Add a replace method to tuples
Add a "replace" method to tuples that returns a new tuple with the element at a given index replaced with a given value. Example implementation: def replace(self, index, value): return self[:index] + (value,) + self[index + 1:] See https://stackoverflow.com/questions/11458239/how-to-change-values-in-a-tuple for more context. Currently, tuples have 2 public methods: index and count. replace would be similarly easy to implement and similarly useful. Furthermore, it would be a natural counterpart to nametuple's _replace method.
This could cause confusion because str.replace() has a completely different API. And indeed if a replace method were added to tuples, a fair case could be made for it having the same API, viz. replace(old, new, count=-1) Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it. Best wishes Rob Cliffe On 10/03/2022 03:42, wfdc via Python-ideas wrote:
This could cause confusion because str.replace() has a completely different API.
We're talking about tuples here, not strings. Saying that a method's API differs for a completely different type, especially when such a difference would be expected given the difference in types, is not a valid objection.
And indeed if a replace method were added to tuples, a fair case could be made for it having the same API, viz. replace(old, new, count=-1)
Not sure what you mean by this. Please clarify.
Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it.
The same can be said for index and count, along with numerous other methods attached to Python's built-in types. Something being simple to implement does not mean it shouldn't be built-in. See Python's "batteries included" philosophy. If users find themselves re-implementing the same utility function over again and over again across different projects, it's a good sign that such a function should be part of the standard library. ------- Original Message ------- On Thursday, March 10th, 2022 at 8:38 PM, Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:
On Fri, 11 Mar 2022 at 14:36, Jeremiah Vivian <nohackingofkrowten@gmail.com> wrote:
On the other hand, it might be an indication that a tuple is the wrong tool for the job. As noted, a namedtuple DOES allow you to replace one component, and to do so by name rather than knowing its index, so possibly that would be a better choice here. ChrisA
On the other hand, it might be an indication that a tuple is the wrong tool for the job.
1. It's not. The original tuple is not being mutated. And it may be desirable to enforce that immutability at the type level. Hence the choice of tuple rather than, say, list. 2. The same "objection" would apply to namedtuple._replace, since namedtuple is also immutable.
As noted, a namedtuple DOES allow you to replace one component, and to do so by name rather than knowing its index, so possibly that would be a better choice here.
Tuples don't have names. They have indices. The tuple equivalent of a namedtuple fieldname is an index. And that's precisely what's being used here. ------- Original Message ------- On Thursday, March 10th, 2022 at 11:13 PM, Chris Angelico <rosuav@gmail.com> wrote:
On the other hand, it might be an indication that a tuple is the wrong tool for the job.
maybe -- if I"m doing a lot of replacing, I likely would turn to a list -- but if you need an immutable, you need an immutable. I'm pretty sure I've written code like: temp = list(a_tuple) temp[i] = new_value a_tuple = tuple(temp) and the OP's suggestion is buggy at the boundary: In [204]: def replace1(tup, index, value): ...: return tup[:index] + (value,) + tup[index+1:] ...: In [205]: tup Out[205]: (1, 2, 3, 4, 5) In [206]: replace1(tup, 5, 100) Out[206]: (1, 2, 3, 4, 5, 100) And converting to a list and back is actually. a tad faster: In [209]: def replace1(tup, index, value): ...: return tup[:index] + (value,) + tup[index+1:] ...: In [210]: def replace2(tup, index, value): ...: temp = list(tup) ...: temp[index] = value ...: return tuple(temp) ...: In [211]: %timeit replace1(tup, 3, 100) 402 ns ± 7.19 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) In [212]: %timeit replace2(tup, 3, 100) 319 ns ± 12.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) Not every one or two line function needs to be in the stdlib, but short functions that are easy to get wrong are good candidates :-)
maybe, but namedtuple is a lot more awkward and heavyweight -- and slower: In [225]: NTup = namedtuple('NTup', ('one', 'two', 'three', 'four', 'five')) In [226]: ntup = NTup(1,2,3,4,5) In [227]: %timeit ntup._replace(three=100) 1.16 µs ± 38.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) I would never recommend a namedtuple for a situation where a tuple is a fine choice other the the lack of a replace method. Frankly, now that dataclasses are built in to the stdlib, I'll bet we'll see fewer uses of namedtuple. Yes, I know they are very different beasts, but I suspect a lot of folks used namedtuples because they wanted a simple data structure with a few attributes -- and didn't need it to be a tuple, or immutable. I may be wrong, it's purely conjecture. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I'm pretty wary of adding methods to builtins -- I wonder how often tuples are either subclassed or duck-typed if the answer isn't "virtually never", then it's a problem to add to the API. That being said, I do find myself wanting to change just one element of a tuple pretty frequently, and it always feels far more awkward than it should. Another possible plus would be optimization: return self[:index] + (value,) + self[index + 1:] Creates three temporary tuples (or maybe four?) in order to make one new one -- that could be far more efficient if built into the C implementation. (unless the interpreter already does some nifty optimizations) -CHB On Thu, Mar 10, 2022 at 7:35 PM Jeremiah Vivian < nohackingofkrowten@gmail.com> wrote:
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Fri, 11 Mar 2022 at 02:20, wfdc via Python-ideas <python-ideas@python.org> wrote:
If users find themselves re-implementing the same utility function over again and over again across different projects, it's a good sign that such a function should be part of the standard library.
And yet you haven't demonstrated that this is the case for your proposal (one Stack Overflow question, with a low number of votes, where it's not clear that the OP shouldn't have been using a list in the first place, isn't particularly compelling evidence). Paul
one Stack Overflow question, with a low number of votes
And yet you haven't demonstrated that this is the case for your
it's not clear that the OP shouldn't have been using a list in
Mind explaining why you say 159 is a "low number of votes" for a StackOverflow question on Python? According to https://stackoverflow.com/questions/tagged/python, this puts it in the top 3031 / 1908740 = 0.00159 = 0.159% of Python questions by vote count. proposal What kind of evidence would satisfy you? And how did previous proposals you supported obtain such evidence? We've already had 2 other participants here attesting to frequent use of this functionality. the first place This has already been explained in this thread. A list is not immutable. A tuple is. Both the old and new tuples are not mutated or mutable, and we want to keep it that way. See namedtuple's ._replace method. namedtuples are also immutable. We simply want the same functionality for tuple. ------- Original Message ------- On Friday, March 11th, 2022 at 4:41 AM, Paul Moore <p.f.moore@gmail.com> wrote:
-1 Not every one line function needs to be a method on a built-in type. I like that tuples have extremely limited methods. Following the iterable protocol seems fine (also indexable). If I were forced to endorse one new method for tuples, I doubt `.replace()` would be in my top five considerations. But if you want it, subclassing is a thing. On Fri, Mar 11, 2022, 2:14 PM wfdc via Python-ideas <python-ideas@python.org> wrote:
Not every one line function needs to be a method on a built-in type.
Not every one line function needs to *not* be a method on a built-in type. See tuple's count method for an example. Again, if users find themselves re-implementing the same utility function over and over again across different projects it's a good sign that such a function should be part of the standard library. Furthermore, other users have already pointed out that my one-liner is not the most *efficient* way to implement it. And it also doesn't include bounds checking.
I like that tuples have extremely limited methods.
Why's that?
Following the iterable protocol seems fine (also indexable).
What does this have to do with the discussion?
If I were forced to endorse one new method for tuples, I doubt `.replace()` would be in my top five considerations.
List the 5 other methods you'd prefer over this one, and explain why you think they're in higher demand and more useful.
But if you want it, subclassing is a thing.
Doesn't solve the problem. It evades the point of why such built-in methods exist in the first place. Again, if users find themselves re-implementing the same utility function over and over again across different projects it's a good sign that such a function should be part of the standard library. ------- Original Message ------- On Friday, March 11th, 2022 at 2:28 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On Fri, Mar 11, 2022, 2:39 PM wfdc <wfdc@protonmail.com> wrote:
Again, if users find themselves re-implementing the same utility function
over and over again across different projects it's a good sign that such a function should be part of the standard library.
I've used Python for 23+ years now. I've had occasion where I'd use this methods maybe in the low-tens of times. I'd call the purpose rare at best. I'm not going to list other one-liners that I'd also not want, but are less rare. There are various, but none I'd advocate adding rather than writing when I need them.
I've used Python for 23+ years now. I've had occasion where I'd use this methods maybe in the low-tens of times. I'd call the purpose rare at best.
This comment is useless without a base rate. What's the base rate for comparable methods that *are* part of the standard library, like index and count?
I'm not going to list other one-liners that I'd also not want, but are less rare. There are various, but none I'd advocate adding rather than writing when I need them.
Why not? If you're not willing explain or back up that comment in any capacity, which should be very easy, what was the point of making it? It's just adding noise to the discussion. ------- Original Message ------- On Friday, March 11th, 2022 at 2:45 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On 2022-03-11 12:03, wfdc via Python-ideas wrote:
You seem to be implicitly suggesting that we should implement all methods that are at least as useful (or frequently needed, or whatever) as existing methods. I have some sympathy with this view, but you seem to be taking it to an unrealistic level. There are many things that have been added to Python over time that are now less useful than other things, for various reasons. We can't just draw a line and say "because .some_method() already exists and has a usefulness score of X, anything that is more useful must now be added" (even if it were possible to define such a usefulness score). Moreover, to do so would only lead to a spiral of madness, since by this logic someone else could come along and say "well, we added .replace() to tuples and I never need that, so there's no reason not to add this other new method most people won't care about or use." As far as I can see, you don't seem to be providing many substantive arguments in favor of your proposal. All you are saying is that sometimes it might be useful, and that there are other existing methods that aren't super useful, so why not add this one too? To me that is sort of like suggesting that it would be good to add a coffee machine and a swimming pool to my car, because my car already has a cigarette lighter and a little storage flap on the sun visor and I don't use those, so there's no reason not to add other random stuff as well. That isn't a very convincing argument. In deciding whether to add some feature to a car, what's relevant is not what other features my car already has that I'm not using, but how much it will cost (in time, money, user confusion, etc.) to add the feature relative to how much benefit it will bring. Similarly, what you are not providing is any consideration of the *costs* of adding this feature relative to the benefit. These costs include implementation and maintenance, increased API surface area for users to familiarize themselves with, and cluttering the type's namespace with yet another method name (even though you seem to acknowledge that it's already cluttered with methods that aren't very useful). Against that we have the benefit of being able to effectively "assign" to a tuple element, which is not zero but also not exactly a burning need. I don't think you're going to get much traction on this suggestion unless you at least engage with this idea of cost and benefits of *your proposed feature* itself, rather than just focusing on what you perceive as the low utility of things that are already part of Python. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
See https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp ------- Original Message ------- On Monday, March 14th, 2022 at 8:39 AM, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
On Fri, 11 Mar 2022 at 19:12, wfdc <wfdc@protonmail.com> wrote:
What kind of evidence would satisfy you? And how did previous proposals you supported obtain such evidence?
Honestly, probably nothing. I don't think this is a good idea in any case, the "not every one-line function needs to be a builtin" principle applies here, in my view. You may have a different view, that's fine. My reason for pointing out that you hadn't demonstrated that your proposal satisfied the criteria that you yourself stated, is that you might find someone more sympathetic to the proposal, and if so, this is the sort of evidence that you'd need in order to take this to python-dev or to create a PR, if you were to have any chance of success. In the past, proposals that succeeded often did so by surveying significant bodies of real-world code (for example, the standard library) and pointing out where the proposed new feature would demonstrably improve the readability or maintainability of parts of the code.
We've already had 2 other participants here attesting to frequent use of this functionality.
You do know how many people use Python? 2 people saying they do something like this isn't particularly significant. Even 2 people out of the number of regular contributors on python-ideas isn't a lot (especially if you take level of participation into account - which is dangerous, because it amounts to an appeal to authority, but how else are you going to demonstrate that a non-trivial number of the millions of people who use Python would find this helpful?
Yes, I understand that if you want to create a new immutable tuple with one value changed, you need to build it from the parts of the original. But there's no *context* here. What's the real-world problem being solved? Why, in the context of that real-world problem, is a tuple (as opposed to, say, an immutable dataclass or namedtuple, both of which have replace methods) the demonstrably best choice, *in spite* of the fact that other choices provide the supposedly important replace functionality? The original SO question sounds suspiciously like an XY-problem to me (see https://en.wikipedia.org/wiki/XY_problem).
See namedtuple's ._replace method. namedtuples are also immutable. We simply want the same functionality for tuple.
I understand *what* you want. But *why*? If the only reason is "for consistency", then fine, that's a reasonable reason. Unlikely to be sufficient in isolation, but that's OK. You asked, you got told "your reasons aren't sufficient". But if you want to persuade someone who has the ability to commit a change to Python to support this proposal, you'll need more (which is what I am trying to help you with). Best of luck in trying to get support for this idea. I'm not keen on it myself, but I'm grateful that you're willing to spend time helping to contribute back towards improving Python. Paul
the "not every one-line function needs to be a builtin" principle applies here, in my view.
The "not every one-line function needs to *not* be a builtin" principle cancels it out. And the "frequently-used functionality should be built-in" (Python's "batteries included" philosophy) overrides it.
you hadn't demonstrated that your proposal satisfied the criteria that you yourself stated
How so?
proposals that succeeded often did so by surveying significant bodies of real-world code (for example, the standard library) and pointing out where the proposed new feature would demonstrably improve the readability or maintainability of parts of the code.
That's something we can work with. Do you have particular examples that demonstrate the kind of evidence that would satisfy you?
You do know how many people use Python? 2 people saying they do something like this isn't particularly significant.
That's strawmanning my point. I said 2 participants *in this thread*. That's a very different population being sampled from to assess significance than "the entire population of Python users".
Even 2 people out of the number of regular contributors on python-ideas isn't a lot
Again, *in this thread*.
especially if you take level of participation into account - which is dangerous, because it amounts to an appeal to authority
I don't see how "it amounts to an appeal to authority". Can you explain? Furthermore, the StackOverflow page provides additional evidence that this functionality is in demand. And it's not just the question itself, but the upvotes, answers, and comments on that page.
Why, in the context of that real-world problem, is a tuple (as opposed to, say, an immutable dataclass or namedtuple, both of which have replace methods) the demonstrably best choice, *in spite* of the fact that other choices provide the supposedly important replace functionality?
See Christopher Barker's reply to Chris Angelico. See also my reply to Chris Angelico.
The original SO question sounds suspiciously like an XY-problem to me (see https://en.wikipedia.org/wiki/XY_problem).
On what basis? And again, it's not just the question itself, but the upvotes, answers, and comments on that page.
I understand *what* you want. But *why*?
See my reply to Chris Angelico. ------- Original Message ------- On Friday, March 11th, 2022 at 2:54 PM, Paul Moore <p.f.moore@gmail.com> wrote:
On Fri, Mar 11, 2022 at 07:12:49PM +0000, wfdc via Python-ideas wrote:
I would think that at least is obvious: the top voted Python question on StackOverflow currently has 11891 votes. This is two orders of magnitude less. It is certainly true that there are over a million Python questions that have received even fewer votes, but 159 votes is still low. In any case, number of votes on StackOverflow is not a good way of judging the worth of a proposal. Votes can be given for many reasons, not just "this proposal should be a built-in function or method". For example, the highest voted question is a request for an explanation, "What does the yield keyword do?". Popularity on StackOverflow is just a vague indication of popularity, not a sign of need or usefulness.
Excellent question! Unfortunately, I think this is a non-trivial change that will require a PEP. (The boundary between trivial and non-trivial is subjective, and others may disagree, but here I feel confident that the core devs will agree that regardless of how trivial the implementation, a change to the API of a core builtin type like tuple will require a PEP.) Yes, this is a lot of trouble to go through, but Python is a 30+ year old mature language. Even though this change alone is not breaking backwards compatibility, it still has consequences. Every change has to justify itself, every feature has costs: - extra code in the interpreter - more things that can break - extra tests - more documentation - more for users to learn - more to remember What may be my essential feature may be your bloat. If the benefits don't outweigh the costs for the majority of people, then the proposal is unlikely to be accepted. You should start by looking at similar successful PEPs, e.g.: Dict union: https://peps.python.org/pep-0584/ String methods to remove prefixes and suffixes: https://peps.python.org/pep-0616/ and maybe some unsuccessful ones as well. By the way, when I started writing PEP 584, it was with the intention that the PEP be rejected once and for all, so we could have a place to point people at when they wanted to know what they can't add dicts. Only it turned out that the case for dict "addition" (dict union) was stronger than anyone thought, and the PEP was accepted. Once you have at least the skeleton of a PEP, you need to find a core developer who thinks it is promising enough to sponsor it. If you can't convince even one single core dev that this proposal has promise, then it is dead in the water. If this was my proposal, I would want to provide a full specification of the API: - Is this part of the tuple class alone? - Or part of the Sequence ABC? Why or why not? - What arguments does it take (index, slice, tuple of indicies)? - Any error conditions, what exceptions will be raised? - Can we use it to delete an item (replace it with nothing at all)? I would want to include: - an overview of what other programming languages provide this and the APIs they use; - and any third-party libraries and classes; - and any builtins and stdlib classes (you already have one, namedtuple); - places in the stdlib that could use this functionality; - examples of your own code where you would use it; - justification for why a simple one-line function isn't sufficient. The last is probably the hardest. We're not immune to efficiency arguments, but generally we don't care about minor inefficiencies. -- Steve
I would think that at least is obvious: the top voted Python question on StackOverflow currently has 11891 votes. This is two orders of magnitude less.
Like I said, this puts it in the top 3031 / 1908740 = 0.00159 = 0.159% of Python questions by vote count. That's not "a low number of votes" no matter how you try to spin it.
Popularity on StackOverflow is just a vague indication of popularity, not a sign of need or usefulness.
Then see https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp for plenty more examples.
Is this part of the tuple class alone? Or part of the Sequence ABC? Why or why not?
Tuple alone, but I'm open to the latter.
What arguments does it take (index, slice, tuple of indicies)?
Index, as specified in the OP. (Again, open to discussion.)
Any error conditions, what exceptions will be raised?
I suggest raising an IndexError when the index is out of range.
Can we use it to delete an item (replace it with nothing at all)?
No.
justification for why a simple one-line function isn't sufficient.
See Christopher Barker's response. (Yes, efficiency is important.) See also the "batteries included" point that was made ages ago. (If users find themselves re-implementing the same utility function over and over again across different projects, it's a good sign that such a function should be part of the standard library.) Your other questions are frankly not necessary to answer and feel like goalpost-moving. The two proposals you held up as examples don't even satisfy them. ------- Original Message ------- On Monday, March 14th, 2022 at 8:32 AM, Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Mar 14, 2022 at 04:38:48PM +0000, wfdc wrote:
It doesn't matter whether it is the top 0.159% or the top 0.00000001%, 160 votes or so is an objectively low number compared to the top ranking questions, with almost 100 times as many votes. Not that it matters. Even if the question had a trillion votes, that alone is not sufficient to justify the proposal.
I got "No results matched your query" on the second URL, at which point I didn't bother with the first. Thank you for providing some search terms, but you misunderstand the process of proposing new functionality to the builtins and stdlib. The onus is not on *us* to do the work, but for those making the proposal. If we ask for examples, we expect you to do the work of curating some good examples, not just to dump a regex in our laps and tell us to search for them ourselves. The bottom line here is that we don't have to justify why the proposal isn't accepted. New feature requests are not Default Accept. All we need to do is *nothing at all* and the proposal doesn't go ahead. At this point, I don't think this discussion on Python-Ideas is going anywhere, which is a shame because I actually do like it. Those who are opposed to changes (whether for good reasons or bad) are dominating the discussion. You aren't going to change their mind. Those who are neutral or in favour have had their voices drowned out. We've already had the mailing list put into Emergency Moderation once, presumably because tempers are getting frayed. I think that your strategies here could include: (1) Give this up as a lost cause. (2) Ignore the negative feedback here and raise it as an enhancement request on the bug tracker, or a PR on the repo. You will *probably* have it rejected and be told to discuss it on Python-Ideas, but you might be lucky and get the interest of a core developer who loves this proposal and accepts it. (But probably not.) (3) Look for some fresh feedback, hopefully (from your perspective) less conservative and negative. Try the "Ideas" topic on Discus. Take into account the feedback you have already received. We've told you the sorts of things that the core devs and Steering Council will be looking for. If you're not willing to provide that, you'll get the same response. Python is a mature language, and the core developers and Steering Council are conservative when it comes to adding marginally useful features that aren't obvious Big Wins, and this is not a Big Win, it is at best a Small Win. This post from one of the core devs may help you understand the culture here: https://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python... My advice to you is that if your time and care factor is low, save your energy for battles you *really* care about. We all want what is best for Python, we may just disagree on what the balance of pros and cons are. If your care factor is high, and you want to continue gathering support for the proposal with the intention of writing a PEP, then listen to those of us who have written successful PEPs in the past.
It is a sign, but how strong a sign is open for debate. My perspective is that replacement of an item in a tuple is *so simple* that it isn't even worth a utility function unless it is generalised. Just build a new tuple using slicing and concatenation expressions. Not every expression is worth an actual named function. I feel that this proposal is *too small* to bother with a function, let alone adding it to the tuple as a builtin method. But generalising it to all sequences, not just tuples, that interests me more. I've used this on strings and lists, probably more often than on tuples. Yes, it raises the bar for acceptance, but it also raises the benefit.
It isn't clear to me what other questions you are referring to. -- Steve
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
Check again in case it's an error. https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp I get 500+ results for each query, including code from several prominent repositories and organizations, such as: - pytorch - pandas - apache - numpy - jax - sympy - cupy - tensorflow - pyqtgraph - aws - dask - deepmind - h5py - cloudflare - facebook research and so on.
My perspective is that replacement of an item in a tuple is so simple that it isn't even worth a utility function unless it is generalised.
I don't oppose the latter.
Just build a new tuple using slicing and concatenation expressions. Not every expression is worth an actual named function. I feel that this proposal is too small to bother with a function, let alone adding it to the tuple as a builtin method.
See Christopher's comments on performance as a builtin. ------- Original Message ------- On Monday, March 14th, 2022 at 7:51 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Mar 15, 2022 at 10:51:21AM +1100, Steven D'Aprano wrote:
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
I tried again, with both URLs, and got plenty of hits this time. Perhaps I had a copy-paste error with the URLs, or a transient error on the website. Those results are promising, but there is a lot of obviously duplicated code there. That's why somebody needs to curate it. -- Steve
On Mon, Mar 14, 2022, 7:52 PM Steven D'Aprano
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
Hmm... while I don't support the proposal, I saw results at both links. Those links were the first good faith comments I saw from OP. A pretty large number of prominent projects use either: foo = bar[:n] + newval + bar[n+1:] Or: newfoo = list(foo) newfoo[n] = newval foo = tuple(newfoo) Somewhere in their codebases. I find that mild evidence in favor of the idea. But the pattern seems to be "occasionally in large-ish projects." On the other hand, looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?" and 25% "this is easy enough as is." None of them made me think "I wish I had a method/function."
Those links were the first good faith comments I saw from OP.
What do you mean by that, exactly?
looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?"
Give several examples. ------- Original Message ------- On Monday, March 14th, 2022 at 11:08 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On Mon, Mar 14, 2022 at 11:08:51PM -0400, David Mertz, Ph.D. wrote:
On the other hand, looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?"
https://fs.blog/chestertons-fence/ Describing this as "silly" seems, well, silly. What else are you to do when you have a record (a, b, c, d) and need a record (a, b, x, d)? There are only two approaches: mutate in place or create a new record. Which one you do depends on whether you have a mutable or immutable record, and which of those you have depends on parts of the code you can't see in your cursory glance. -- Steve
On 11/03/2022 02:19, wfdc wrote: proposed method.
A function that replaced tuple elements by value rather than by index would no doubt be useful to somebody (not that I'm proposing it). So it could use the same API as str.replace().
Hm, off the top of my head, I could (I believe) write "count" as an (inefficient) 1-liner, but not "index". I suggest it's harder than you think. (Try it!)
"Not every 1-line function needs to be a built-in".
Well, you are 1 user. Have you evidence that there are (many) others? Best wishes Rob Cliffe
But *humans* can be confused by "replace" having a totally different API in different contexts.
I doubt that's the case here. The closest equivalent to tuple's .replace method would be namedtuple's _.replace method, which also has a different API from string's .replace method.
I could (I believe) write "count" as an (inefficient) 1-liner, but not "index". I suggest it's harder than you think. (Try it!)
How much harder? Can you post your candidate? In any case, my point still stands.
"Not every 1-line function needs to be a built-in".
Not every 1-line function needs to *not* be a built-in.
Well, you are 1 user. Have you evidence that there are (many) others? See the StackOverflow link and the 2 other participants in this thread who attested to frequent use of this functionality.
------- Original Message ------- On Friday, March 11th, 2022 at 6:30 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
On Sat, 12 Mar 2022 at 06:33, wfdc via Python-ideas <python-ideas@python.org> wrote:
You still haven't shown why a namedtuple is wrong for your use-case. In fact, you haven't shown anything of your use-case, other than that you've written a one-liner and wish that it were a method. What is the larger context in which this is such an incredibly common operation? In fact, if it's really such a frequent need, maybe you and/or other participants can show more than one use-case. That would be helpful in understanding why tuples need this as a method. ChrisA
You still haven't shown why a namedtuple is wrong for your use-case.
See Christopher Barker's previous reply to you. Furthermore, namedtuples have *fieldnames*. Tuples have *indices*. Tuples are conceptually more appropriate if we're dealing with what are supposed to be numeric indices into some sequential datastructure, rather than a dictionary-like one.
In fact, you haven't shown anything of your use-case, other than that you've written a one-liner and wish that it were a method. What is the larger context in which this is such an incredibly common operation?
1. As already pointed out in the thread, the one-liner is not the most efficient way to implement it, nor does it do bounds checking. 2. See the StackOverflow link and the 2 other participants in this thread attesting to frequent use of this functionality. ------- Original Message ------- On Friday, March 11th, 2022 at 2:36 PM, Chris Angelico <rosuav@gmail.com> wrote:
you haven't shown us what your use-case actually is
Any use-case where you'd want to modify an entry of an immutable sequence. Modifying an immutable datastructure is not a contradictory statement. In fact, there's a whole literature on it. See https://en.wikipedia.org/wiki/Purely_functional_data_structure https://en.wikipedia.org/wiki/Persistent_data_structure As Marco Sulla pointed, out, "Performance apart, this will allow you to code in functional style more easily." namedtuple's ._replace method is an existing example (as is record updating in Haskell, which is a purely functional language, with the attending benefits of immutability). For me *specifically*, my latest use-case is in a game-theoretic setting where the tuple represents a strategy profile. But the details of that would take us far beyond the scope of the discussion.
whether it would actually be more appropriate for a list instead
Lists are not immutable, so they fail the criteria.
Do all your tuples have the same length?
In an inner context, yes. But in an outer context, no. What I mean by that is that, in my particular case, the number of entries is a hyperparameter. ------- Original Message ------- On Friday, March 11th, 2022 at 2:56 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 12 Mar 2022 at 07:38, wfdc <wfdc@protonmail.com> wrote:
you haven't shown us what your use-case actually is
Any use-case where you'd want to modify an entry of an immutable sequence.
That doesn't answer the question any more than your original code does. You're still starting with the assumption that you want to use a tuple, and using that as proof that you need to use a tuple. Show us your ACTUAL use case, your actual code, and why you can't use either a list or a namedtuple. Stop assuming that you are correct before you begin, because *we don't agree* that a tuple is necessary. You have not shown us any code, only made this repeated assertion and then wondered why we don't believe you.
Yes, and dataclasses and namedtuples support this concept very nicely. You still haven't shown why neither works.
I don't understand how a strategy profile has to be a tuple. In fact, particularly since you're wanting to replace one part of it, it definitely seems like a good use-case for a dataclass.
whether it would actually be more appropriate for a list instead
Lists are not immutable, so they fail the criteria.
Why? Do you see how you are still failing to show any actual code, and thus any actual demonstration of why a tuple is necessary? Repeatedly stating that it has to be immutable, has to not be a namedtuple, has to not be a dataclass, and has to have a replace method, doesn't make it true. ChrisA
See https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp and https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp
Yes, and dataclasses and namedtuples support this concept very nicely. You still haven't shown why neither works.
1. They're less efficient, as Christopher pointed out long ago. 2. They're not sequences, so conceptually the wrong tool for the job.
it definitely seems like a good use-case for a dataclass.
See above.
Repeatedly stating that it has to be immutable, has to not be a namedtuple, has to not be a dataclass, and has to have a replace method, doesn't make it true.
See above. ------- Original Message ------- On Monday, March 14th, 2022 at 8:30 AM, Chris Angelico <rosuav@gmail.com> wrote:
A namedtuple IS a sequence. It is a tuple. They are deliberately designed to be drop-in replacements for tuples.
I meant that they're not meant to be addressed as sequences. That's the whole point of namedtuple: to avoid doing that. And the first point still applies. ------- Original Message ------- On Monday, March 14th, 2022 at 12:19 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, 15 Mar 2022 at 04:02, wfdc <wfdc@protonmail.com> wrote:
A namedtuple IS a sequence. It is a tuple. They are deliberately designed to be drop-in replacements for tuples.
I meant that they're not meant to be addressed as sequences. That's the whole point of namedtuple: to avoid doing that.
The whole point is that they are both sequences and structures. https://docs.python.org/3/library/collections.html#collections.namedtuple ChrisA
The whole *point* is that namedtuples let you use *named fields* rather than indices. That's the point. That's the purpose. What are the named fields supposed to be if the datastructure is being treated as a sequence? Indices again? Then what's the point of using namedtuple rather than tuple in the first place? Again, I direct you to Christopher's comment on the matter. ------- Original Message ------- On Monday, March 14th, 2022 at 1:07 PM, Chris Angelico <rosuav@gmail.com> wrote:
On 15/03/22 6:14 am, wfdc via Python-ideas wrote:
The whole *point* is that namedtuples let you use *named fields* rather than indices. That's the point. That's the purpose.
I would say the point of namedtuple is to let you use either field names or indices, whichever makes the most sense for what you're doing at the time. Otherwise why not just use a regular class? -- Greg
On Mon, 14 Mar 2022 at 22:21, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
As I understood it, one motivation for the introduction of namedtuple was that where you had an API that returned a tuple you could change that to return a namedtuple without breaking backwards compatibility. Then older code expecting a tuple could still use tuple-ish idioms like unpacking, slicing or indexing but newer code could use names to access the elements. -- Oscar
See my previous response to Chris:
What are the named fields supposed to be if the datastructure is being treated as a sequence? Indices again? Then what's the point of using namedtuple rather than tuple in the first place?
And, by the way, field names must be valid identifiers.
So again, what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length?
------- Original Message ------- On Monday, March 14th, 2022 at 6:19 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On Tue, 15 Mar 2022 at 10:02, wfdc via Python-ideas <python-ideas@python.org> wrote:
It's very hard to answer a question that is fundamentally based on a misunderstanding of namedtuples. Since a namedtuple *is* supposed to be both a sequence and a record, the names are part of the record interface, and the sequence is still part of that. Also, the field names don't have to be valid identifiers (although they usually will). But if it's a sequence of arbitrary length, then why is it a tuple at all? You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple. Do you see why we've been asking you to SHOW YOUR CODE? I am done. I'm not going to further respond in this thread until there is some actual code to discuss. ChrisA
Again, you neglected to answer the question:
what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length?
Since a namedtuple is supposed to be both a sequence and a record, the names are part of the record interface, and the sequence is still part of that.
The point is that I *don't* want to treat it as a record, and I have no reason to. That's precisely the point I'm making!
the field names don't have to be valid identifiers
Yes, they do.
if it's a sequence of arbitrary length, then why is it a tuple at all?
Why wouldn't it be? A tuple is a sequence. And a sequence can be of arbitrary length.
You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple.
See my first question.
Do you see why we've been asking you to SHOW YOUR CODE?
I already provided *plenty* of examples through the Sourcegraph queries. There's no reason for me to give you my specific code, nor does it help this discussion in any way, since I already gave you plenty of examples. ------- Original Message ------- On Monday, March 14th, 2022 at 7:04 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Mar 15, 2022 at 10:04:45AM +1100, Chris Angelico wrote:
But if it's a sequence of arbitrary length, then why is it a tuple at all?
def func(p, q, *sequence_of_arbitrary_length): ... I guess that the interpreter is Doing It All Wrong and making an XY Problem mistake. Right?
You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple.
To be precise, namedtuples are a subclass of tuple. For many purposes, we can treat namedtuples and tuples the same way. But namedtuples **extend the tuple API**. In terms of the Liskov Substitution Principle, we can say that any place you use a tuple, you can replace it with a namedtuple, but not the other way around. However, in *practical* terms, the LSP is not the full truth. The problem here is easily seen by considering the *args tuple from function definitions again. If we choose to replace the plain tuple with a namedtuple, what meaningful names would you use for the fields? Aside from the practical problem that namedtuple() doesn't support arbitrary length records, there is no meaningful names for the fields. The best we can do is some mapping of index to ordinals: zeroeth, first, second, third, fourth, fifth, ... but that's hardly meaningful in the sense intended. So namedtuple is a red herring here and the LSP is not the whole truth. Despite what Liskov says, plain vanilla tuples can be considered a superset of named tuples, not a subset: - tuples support records of arbitrary length; - tuples support anonymous fields. The bottom line here is this: There is nothing wrong with using plain tuples as your data structure, and the instance from certain people that wfdc is wrong to do so is arrogant, rude and condescending. -- Steve
And, by the way, field names must be valid identifiers. So again, what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length? ------- Original Message ------- On Monday, March 14th, 2022 at 1:07 PM, Chris Angelico <rosuav@gmail.com> wrote:
How about something like def index(self, x): return next(i for i, a in enumerate(self) if a == x) Including start and end: def index(self, x, start=0, end=-1): return next(i for i, a in tuple(enumerate(self))[start:end] if a == x) ------- Original Message ------- On Friday, March 11th, 2022 at 4:02 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
Please. If we're now talking about specific exceptions, my replace isn't a one-liner either. I was clearly referring to actual functionality. If you want that specific exception, 2 lines are sufficient. ------- Original Message ------- On Friday, March 11th, 2022 at 4:45 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
On Fri, Mar 11, 2022 at 01:38:57AM +0000, Rob Cliffe via Python-ideas wrote:
This could cause confusion because str.replace() has a completely different API.
The old "painter.draw()" versus "gun_slinger.draw()" problem. It does exist, but it's not generally a *major* problem.
A reasonable suggestion. Does replace change items according to their position, or their value? Perhaps a better name would help. wfdc (should we call you something which is not a random-looking string of consonants?) it might help if you could give examples of this functionality from other languages or classes. What do they call the method? It would be nice to have a better API for this function. I have used it many times, for strings, lists and tuples, it is really a generate sequence operation: * make a copy of the sequence, replacing the item at position p with a new value; We can do it with type-specific expressions: seq[:p] + [value] + seq[p+1:] seq[:p] + (value,) + seq[p+1:] string[:p] + "c" + string[p+1:] but there is no good, type-agnostic way to easily do it. Wacky thought: what if index assignment was an expression? function(arg, seq[p]=value, another_arg) would pass a copy of seq with the item at p replaced by value. That would work with slices as well: seq[a:b] = values Now that I have seen this syntax, I want it!!! (It is not clear to me if this would be compatible to the existing `__setitem__ API.)
Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it.
Sure, not all one-liners need to be built-in. But some do. -- Steve
wfdc (should we call you something which is not a random-looking string of consonants?)
wfdc is fine.
it might help if you could give examples of this functionality from other languages or classes. What do they call the method?
https://clojuredocs.org/clojure.core/assoc https://immutable-js.com/docs/v4.0.0/set()/ http://swannodette.github.io/mori/#assoc https://hackage.haskell.org/package/lens-5.1/docs/Control-Lens-Operators.htm... https://hackage.haskell.org/package/containers-0.6.5.1/docs/Data-Sequence.ht... What about 'update' or 'set'?
what if index assignment was an expression? [...] Now that I have seen this syntax, I want it!!!
Interesting idea. Perhaps we can discuss it further. ------- Original Message ------- On Monday, March 14th, 2022 at 8:38 AM, Steven D'Aprano <steve@pearwood.info> wrote:
jax is another example that uses the name 'set' for this functionality: https://jax.readthedocs.io/en/latest/jax.numpy.html On second thought, perhaps it wouldn't be a bad idea to extend this proposal to built-in sequence types in general (list, str, tuple, and bytes). Examples of this functionality being used on those other types: https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B.*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp If extended beyond tuples, we would have to address the name collision with the existing 'replace' methods of str and bytes. In that case, something like 'set' or 'update' might work. (I prefer the former for its conciseness.) ------- Original Message ------- On Monday, March 14th, 2022 at 12:47 PM, wfdc <wfdc@protonmail.com> wrote:
On Mon, Mar 14, 2022 at 11:16 AM wfdc via Python-ideas < python-ideas@python.org> wrote:
jax is another example that uses the name 'set' for this functionality:
set() strikes me as a better name for a mutating operation. Apparently Jax arrays are immutable, but from the Jax docs: "For example, instead of in-place array updates (x[i] = y), JAX provides an alternative pure indexed update function x.at[i].set(y)" I'd need to read more before I understood that that means (what does x.at[] return? But this does look pretty different than the proposal. Personally, I think the symmetry between this and datatime.replace and namedtuple._replace is actually a good thing, even though the API is a little different. On second thought, perhaps it wouldn't be a bad idea to extend this
proposal to built-in sequence types in general (list, str, tuple, and bytes).
I don't think it should be added to mutable types -- there are already ways to do that, and there are any number of "make a new one with some change" methods that mutables don't have. Essentially, the API of mutable and immutable types is already different. Are you proposing that this be added to an ABC? That would be a heavier lift, and now that I think about it, there is no ABC for immutable sequences -- they are simply sequences -- which don't have any mutating methods. If you are proposing that it be added to a handful of built-in types -- I don't think that should be done for the sake of symmetry, but only if it's useful for that type. -- do we often want to replace the nth character in a string? or the nth byte in a bytes object? Maybe? I don't think I"ve ever needed to do that.
yup -- so I suppose another name would be helpful for future extending. (dict has update() -- would that add confusion?) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Mon, Mar 14, 2022 at 11:53:37AM -0700, Christopher Barker wrote:
How do we make a new list with a change other than the same slice and concatenation we use with tuples? alist[:i] + [value] + alist[i+1:] I mean as an expression, of course we can split it over two statements: newlist = alist[:] newlist[i] = value which is fine, but it does lose the expressiveness of an expression :-) -- Steve
See the following Sourcegraph query for examples of use of this pattern: https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp Regex: \[\s*:\s*(i|j|k|ind|index)\s*\]\s*\+\s*\(.*\,\)\s*\+\s*[\w\.]+\[\s*(i|j|k|ind|index)\s*\+\s*1\s*:\s*\] ------- Original Message ------- On Wednesday, March 9th, 2022 at 10:42 PM, wfdc <wfdc@protonmail.com> wrote:
See also the following Sourcegraph query for examples of use of the second approach: https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp Regex: =\s*list\([\w\.]+\)\s*\w+\[\w+\]\s*=\s*[\w\.]+\s*[\w\.]+\s*=\s*tuple\(\w+\) ------- Original Message ------- On Friday, March 11th, 2022 at 7:36 PM, wfdc <wfdc@protonmail.com> wrote:
On 2022-03-09 19:42, wfdc via Python-ideas wrote:
Add a "replace" method to tuples that returnsa new tuple with the element at a given index replaced with a given value.
Hmm, no one appears to have mentioned DateTime.replace(). Did I miss it? https://docs.python.org/3/library/datetime.html#datetime.date.replace I think it demonstrates that this is a valid and useful pattern. Agree that I don't often have a need, but am glad it exists. -Mike
This could cause confusion because str.replace() has a completely different API. And indeed if a replace method were added to tuples, a fair case could be made for it having the same API, viz. replace(old, new, count=-1) Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it. Best wishes Rob Cliffe On 10/03/2022 03:42, wfdc via Python-ideas wrote:
This could cause confusion because str.replace() has a completely different API.
We're talking about tuples here, not strings. Saying that a method's API differs for a completely different type, especially when such a difference would be expected given the difference in types, is not a valid objection.
And indeed if a replace method were added to tuples, a fair case could be made for it having the same API, viz. replace(old, new, count=-1)
Not sure what you mean by this. Please clarify.
Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it.
The same can be said for index and count, along with numerous other methods attached to Python's built-in types. Something being simple to implement does not mean it shouldn't be built-in. See Python's "batteries included" philosophy. If users find themselves re-implementing the same utility function over again and over again across different projects, it's a good sign that such a function should be part of the standard library. ------- Original Message ------- On Thursday, March 10th, 2022 at 8:38 PM, Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:
On Fri, 11 Mar 2022 at 14:36, Jeremiah Vivian <nohackingofkrowten@gmail.com> wrote:
On the other hand, it might be an indication that a tuple is the wrong tool for the job. As noted, a namedtuple DOES allow you to replace one component, and to do so by name rather than knowing its index, so possibly that would be a better choice here. ChrisA
On the other hand, it might be an indication that a tuple is the wrong tool for the job.
1. It's not. The original tuple is not being mutated. And it may be desirable to enforce that immutability at the type level. Hence the choice of tuple rather than, say, list. 2. The same "objection" would apply to namedtuple._replace, since namedtuple is also immutable.
As noted, a namedtuple DOES allow you to replace one component, and to do so by name rather than knowing its index, so possibly that would be a better choice here.
Tuples don't have names. They have indices. The tuple equivalent of a namedtuple fieldname is an index. And that's precisely what's being used here. ------- Original Message ------- On Thursday, March 10th, 2022 at 11:13 PM, Chris Angelico <rosuav@gmail.com> wrote:
On the other hand, it might be an indication that a tuple is the wrong tool for the job.
maybe -- if I"m doing a lot of replacing, I likely would turn to a list -- but if you need an immutable, you need an immutable. I'm pretty sure I've written code like: temp = list(a_tuple) temp[i] = new_value a_tuple = tuple(temp) and the OP's suggestion is buggy at the boundary: In [204]: def replace1(tup, index, value): ...: return tup[:index] + (value,) + tup[index+1:] ...: In [205]: tup Out[205]: (1, 2, 3, 4, 5) In [206]: replace1(tup, 5, 100) Out[206]: (1, 2, 3, 4, 5, 100) And converting to a list and back is actually. a tad faster: In [209]: def replace1(tup, index, value): ...: return tup[:index] + (value,) + tup[index+1:] ...: In [210]: def replace2(tup, index, value): ...: temp = list(tup) ...: temp[index] = value ...: return tuple(temp) ...: In [211]: %timeit replace1(tup, 3, 100) 402 ns ± 7.19 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) In [212]: %timeit replace2(tup, 3, 100) 319 ns ± 12.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) Not every one or two line function needs to be in the stdlib, but short functions that are easy to get wrong are good candidates :-)
maybe, but namedtuple is a lot more awkward and heavyweight -- and slower: In [225]: NTup = namedtuple('NTup', ('one', 'two', 'three', 'four', 'five')) In [226]: ntup = NTup(1,2,3,4,5) In [227]: %timeit ntup._replace(three=100) 1.16 µs ± 38.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) I would never recommend a namedtuple for a situation where a tuple is a fine choice other the the lack of a replace method. Frankly, now that dataclasses are built in to the stdlib, I'll bet we'll see fewer uses of namedtuple. Yes, I know they are very different beasts, but I suspect a lot of folks used namedtuples because they wanted a simple data structure with a few attributes -- and didn't need it to be a tuple, or immutable. I may be wrong, it's purely conjecture. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I'm pretty wary of adding methods to builtins -- I wonder how often tuples are either subclassed or duck-typed if the answer isn't "virtually never", then it's a problem to add to the API. That being said, I do find myself wanting to change just one element of a tuple pretty frequently, and it always feels far more awkward than it should. Another possible plus would be optimization: return self[:index] + (value,) + self[index + 1:] Creates three temporary tuples (or maybe four?) in order to make one new one -- that could be far more efficient if built into the C implementation. (unless the interpreter already does some nifty optimizations) -CHB On Thu, Mar 10, 2022 at 7:35 PM Jeremiah Vivian < nohackingofkrowten@gmail.com> wrote:
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Fri, 11 Mar 2022 at 02:20, wfdc via Python-ideas <python-ideas@python.org> wrote:
If users find themselves re-implementing the same utility function over again and over again across different projects, it's a good sign that such a function should be part of the standard library.
And yet you haven't demonstrated that this is the case for your proposal (one Stack Overflow question, with a low number of votes, where it's not clear that the OP shouldn't have been using a list in the first place, isn't particularly compelling evidence). Paul
one Stack Overflow question, with a low number of votes
And yet you haven't demonstrated that this is the case for your
it's not clear that the OP shouldn't have been using a list in
Mind explaining why you say 159 is a "low number of votes" for a StackOverflow question on Python? According to https://stackoverflow.com/questions/tagged/python, this puts it in the top 3031 / 1908740 = 0.00159 = 0.159% of Python questions by vote count. proposal What kind of evidence would satisfy you? And how did previous proposals you supported obtain such evidence? We've already had 2 other participants here attesting to frequent use of this functionality. the first place This has already been explained in this thread. A list is not immutable. A tuple is. Both the old and new tuples are not mutated or mutable, and we want to keep it that way. See namedtuple's ._replace method. namedtuples are also immutable. We simply want the same functionality for tuple. ------- Original Message ------- On Friday, March 11th, 2022 at 4:41 AM, Paul Moore <p.f.moore@gmail.com> wrote:
-1 Not every one line function needs to be a method on a built-in type. I like that tuples have extremely limited methods. Following the iterable protocol seems fine (also indexable). If I were forced to endorse one new method for tuples, I doubt `.replace()` would be in my top five considerations. But if you want it, subclassing is a thing. On Fri, Mar 11, 2022, 2:14 PM wfdc via Python-ideas <python-ideas@python.org> wrote:
Not every one line function needs to be a method on a built-in type.
Not every one line function needs to *not* be a method on a built-in type. See tuple's count method for an example. Again, if users find themselves re-implementing the same utility function over and over again across different projects it's a good sign that such a function should be part of the standard library. Furthermore, other users have already pointed out that my one-liner is not the most *efficient* way to implement it. And it also doesn't include bounds checking.
I like that tuples have extremely limited methods.
Why's that?
Following the iterable protocol seems fine (also indexable).
What does this have to do with the discussion?
If I were forced to endorse one new method for tuples, I doubt `.replace()` would be in my top five considerations.
List the 5 other methods you'd prefer over this one, and explain why you think they're in higher demand and more useful.
But if you want it, subclassing is a thing.
Doesn't solve the problem. It evades the point of why such built-in methods exist in the first place. Again, if users find themselves re-implementing the same utility function over and over again across different projects it's a good sign that such a function should be part of the standard library. ------- Original Message ------- On Friday, March 11th, 2022 at 2:28 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On Fri, Mar 11, 2022, 2:39 PM wfdc <wfdc@protonmail.com> wrote:
Again, if users find themselves re-implementing the same utility function
over and over again across different projects it's a good sign that such a function should be part of the standard library.
I've used Python for 23+ years now. I've had occasion where I'd use this methods maybe in the low-tens of times. I'd call the purpose rare at best. I'm not going to list other one-liners that I'd also not want, but are less rare. There are various, but none I'd advocate adding rather than writing when I need them.
I've used Python for 23+ years now. I've had occasion where I'd use this methods maybe in the low-tens of times. I'd call the purpose rare at best.
This comment is useless without a base rate. What's the base rate for comparable methods that *are* part of the standard library, like index and count?
I'm not going to list other one-liners that I'd also not want, but are less rare. There are various, but none I'd advocate adding rather than writing when I need them.
Why not? If you're not willing explain or back up that comment in any capacity, which should be very easy, what was the point of making it? It's just adding noise to the discussion. ------- Original Message ------- On Friday, March 11th, 2022 at 2:45 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On 2022-03-11 12:03, wfdc via Python-ideas wrote:
You seem to be implicitly suggesting that we should implement all methods that are at least as useful (or frequently needed, or whatever) as existing methods. I have some sympathy with this view, but you seem to be taking it to an unrealistic level. There are many things that have been added to Python over time that are now less useful than other things, for various reasons. We can't just draw a line and say "because .some_method() already exists and has a usefulness score of X, anything that is more useful must now be added" (even if it were possible to define such a usefulness score). Moreover, to do so would only lead to a spiral of madness, since by this logic someone else could come along and say "well, we added .replace() to tuples and I never need that, so there's no reason not to add this other new method most people won't care about or use." As far as I can see, you don't seem to be providing many substantive arguments in favor of your proposal. All you are saying is that sometimes it might be useful, and that there are other existing methods that aren't super useful, so why not add this one too? To me that is sort of like suggesting that it would be good to add a coffee machine and a swimming pool to my car, because my car already has a cigarette lighter and a little storage flap on the sun visor and I don't use those, so there's no reason not to add other random stuff as well. That isn't a very convincing argument. In deciding whether to add some feature to a car, what's relevant is not what other features my car already has that I'm not using, but how much it will cost (in time, money, user confusion, etc.) to add the feature relative to how much benefit it will bring. Similarly, what you are not providing is any consideration of the *costs* of adding this feature relative to the benefit. These costs include implementation and maintenance, increased API surface area for users to familiarize themselves with, and cluttering the type's namespace with yet another method name (even though you seem to acknowledge that it's already cluttered with methods that aren't very useful). Against that we have the benefit of being able to effectively "assign" to a tuple element, which is not zero but also not exactly a burning need. I don't think you're going to get much traction on this suggestion unless you at least engage with this idea of cost and benefits of *your proposed feature* itself, rather than just focusing on what you perceive as the low utility of things that are already part of Python. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
See https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp ------- Original Message ------- On Monday, March 14th, 2022 at 8:39 AM, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
On Fri, 11 Mar 2022 at 19:12, wfdc <wfdc@protonmail.com> wrote:
What kind of evidence would satisfy you? And how did previous proposals you supported obtain such evidence?
Honestly, probably nothing. I don't think this is a good idea in any case, the "not every one-line function needs to be a builtin" principle applies here, in my view. You may have a different view, that's fine. My reason for pointing out that you hadn't demonstrated that your proposal satisfied the criteria that you yourself stated, is that you might find someone more sympathetic to the proposal, and if so, this is the sort of evidence that you'd need in order to take this to python-dev or to create a PR, if you were to have any chance of success. In the past, proposals that succeeded often did so by surveying significant bodies of real-world code (for example, the standard library) and pointing out where the proposed new feature would demonstrably improve the readability or maintainability of parts of the code.
We've already had 2 other participants here attesting to frequent use of this functionality.
You do know how many people use Python? 2 people saying they do something like this isn't particularly significant. Even 2 people out of the number of regular contributors on python-ideas isn't a lot (especially if you take level of participation into account - which is dangerous, because it amounts to an appeal to authority, but how else are you going to demonstrate that a non-trivial number of the millions of people who use Python would find this helpful?
Yes, I understand that if you want to create a new immutable tuple with one value changed, you need to build it from the parts of the original. But there's no *context* here. What's the real-world problem being solved? Why, in the context of that real-world problem, is a tuple (as opposed to, say, an immutable dataclass or namedtuple, both of which have replace methods) the demonstrably best choice, *in spite* of the fact that other choices provide the supposedly important replace functionality? The original SO question sounds suspiciously like an XY-problem to me (see https://en.wikipedia.org/wiki/XY_problem).
See namedtuple's ._replace method. namedtuples are also immutable. We simply want the same functionality for tuple.
I understand *what* you want. But *why*? If the only reason is "for consistency", then fine, that's a reasonable reason. Unlikely to be sufficient in isolation, but that's OK. You asked, you got told "your reasons aren't sufficient". But if you want to persuade someone who has the ability to commit a change to Python to support this proposal, you'll need more (which is what I am trying to help you with). Best of luck in trying to get support for this idea. I'm not keen on it myself, but I'm grateful that you're willing to spend time helping to contribute back towards improving Python. Paul
the "not every one-line function needs to be a builtin" principle applies here, in my view.
The "not every one-line function needs to *not* be a builtin" principle cancels it out. And the "frequently-used functionality should be built-in" (Python's "batteries included" philosophy) overrides it.
you hadn't demonstrated that your proposal satisfied the criteria that you yourself stated
How so?
proposals that succeeded often did so by surveying significant bodies of real-world code (for example, the standard library) and pointing out where the proposed new feature would demonstrably improve the readability or maintainability of parts of the code.
That's something we can work with. Do you have particular examples that demonstrate the kind of evidence that would satisfy you?
You do know how many people use Python? 2 people saying they do something like this isn't particularly significant.
That's strawmanning my point. I said 2 participants *in this thread*. That's a very different population being sampled from to assess significance than "the entire population of Python users".
Even 2 people out of the number of regular contributors on python-ideas isn't a lot
Again, *in this thread*.
especially if you take level of participation into account - which is dangerous, because it amounts to an appeal to authority
I don't see how "it amounts to an appeal to authority". Can you explain? Furthermore, the StackOverflow page provides additional evidence that this functionality is in demand. And it's not just the question itself, but the upvotes, answers, and comments on that page.
Why, in the context of that real-world problem, is a tuple (as opposed to, say, an immutable dataclass or namedtuple, both of which have replace methods) the demonstrably best choice, *in spite* of the fact that other choices provide the supposedly important replace functionality?
See Christopher Barker's reply to Chris Angelico. See also my reply to Chris Angelico.
The original SO question sounds suspiciously like an XY-problem to me (see https://en.wikipedia.org/wiki/XY_problem).
On what basis? And again, it's not just the question itself, but the upvotes, answers, and comments on that page.
I understand *what* you want. But *why*?
See my reply to Chris Angelico. ------- Original Message ------- On Friday, March 11th, 2022 at 2:54 PM, Paul Moore <p.f.moore@gmail.com> wrote:
On Fri, Mar 11, 2022 at 07:12:49PM +0000, wfdc via Python-ideas wrote:
I would think that at least is obvious: the top voted Python question on StackOverflow currently has 11891 votes. This is two orders of magnitude less. It is certainly true that there are over a million Python questions that have received even fewer votes, but 159 votes is still low. In any case, number of votes on StackOverflow is not a good way of judging the worth of a proposal. Votes can be given for many reasons, not just "this proposal should be a built-in function or method". For example, the highest voted question is a request for an explanation, "What does the yield keyword do?". Popularity on StackOverflow is just a vague indication of popularity, not a sign of need or usefulness.
Excellent question! Unfortunately, I think this is a non-trivial change that will require a PEP. (The boundary between trivial and non-trivial is subjective, and others may disagree, but here I feel confident that the core devs will agree that regardless of how trivial the implementation, a change to the API of a core builtin type like tuple will require a PEP.) Yes, this is a lot of trouble to go through, but Python is a 30+ year old mature language. Even though this change alone is not breaking backwards compatibility, it still has consequences. Every change has to justify itself, every feature has costs: - extra code in the interpreter - more things that can break - extra tests - more documentation - more for users to learn - more to remember What may be my essential feature may be your bloat. If the benefits don't outweigh the costs for the majority of people, then the proposal is unlikely to be accepted. You should start by looking at similar successful PEPs, e.g.: Dict union: https://peps.python.org/pep-0584/ String methods to remove prefixes and suffixes: https://peps.python.org/pep-0616/ and maybe some unsuccessful ones as well. By the way, when I started writing PEP 584, it was with the intention that the PEP be rejected once and for all, so we could have a place to point people at when they wanted to know what they can't add dicts. Only it turned out that the case for dict "addition" (dict union) was stronger than anyone thought, and the PEP was accepted. Once you have at least the skeleton of a PEP, you need to find a core developer who thinks it is promising enough to sponsor it. If you can't convince even one single core dev that this proposal has promise, then it is dead in the water. If this was my proposal, I would want to provide a full specification of the API: - Is this part of the tuple class alone? - Or part of the Sequence ABC? Why or why not? - What arguments does it take (index, slice, tuple of indicies)? - Any error conditions, what exceptions will be raised? - Can we use it to delete an item (replace it with nothing at all)? I would want to include: - an overview of what other programming languages provide this and the APIs they use; - and any third-party libraries and classes; - and any builtins and stdlib classes (you already have one, namedtuple); - places in the stdlib that could use this functionality; - examples of your own code where you would use it; - justification for why a simple one-line function isn't sufficient. The last is probably the hardest. We're not immune to efficiency arguments, but generally we don't care about minor inefficiencies. -- Steve
I would think that at least is obvious: the top voted Python question on StackOverflow currently has 11891 votes. This is two orders of magnitude less.
Like I said, this puts it in the top 3031 / 1908740 = 0.00159 = 0.159% of Python questions by vote count. That's not "a low number of votes" no matter how you try to spin it.
Popularity on StackOverflow is just a vague indication of popularity, not a sign of need or usefulness.
Then see https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp for plenty more examples.
Is this part of the tuple class alone? Or part of the Sequence ABC? Why or why not?
Tuple alone, but I'm open to the latter.
What arguments does it take (index, slice, tuple of indicies)?
Index, as specified in the OP. (Again, open to discussion.)
Any error conditions, what exceptions will be raised?
I suggest raising an IndexError when the index is out of range.
Can we use it to delete an item (replace it with nothing at all)?
No.
justification for why a simple one-line function isn't sufficient.
See Christopher Barker's response. (Yes, efficiency is important.) See also the "batteries included" point that was made ages ago. (If users find themselves re-implementing the same utility function over and over again across different projects, it's a good sign that such a function should be part of the standard library.) Your other questions are frankly not necessary to answer and feel like goalpost-moving. The two proposals you held up as examples don't even satisfy them. ------- Original Message ------- On Monday, March 14th, 2022 at 8:32 AM, Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Mar 14, 2022 at 04:38:48PM +0000, wfdc wrote:
It doesn't matter whether it is the top 0.159% or the top 0.00000001%, 160 votes or so is an objectively low number compared to the top ranking questions, with almost 100 times as many votes. Not that it matters. Even if the question had a trillion votes, that alone is not sufficient to justify the proposal.
I got "No results matched your query" on the second URL, at which point I didn't bother with the first. Thank you for providing some search terms, but you misunderstand the process of proposing new functionality to the builtins and stdlib. The onus is not on *us* to do the work, but for those making the proposal. If we ask for examples, we expect you to do the work of curating some good examples, not just to dump a regex in our laps and tell us to search for them ourselves. The bottom line here is that we don't have to justify why the proposal isn't accepted. New feature requests are not Default Accept. All we need to do is *nothing at all* and the proposal doesn't go ahead. At this point, I don't think this discussion on Python-Ideas is going anywhere, which is a shame because I actually do like it. Those who are opposed to changes (whether for good reasons or bad) are dominating the discussion. You aren't going to change their mind. Those who are neutral or in favour have had their voices drowned out. We've already had the mailing list put into Emergency Moderation once, presumably because tempers are getting frayed. I think that your strategies here could include: (1) Give this up as a lost cause. (2) Ignore the negative feedback here and raise it as an enhancement request on the bug tracker, or a PR on the repo. You will *probably* have it rejected and be told to discuss it on Python-Ideas, but you might be lucky and get the interest of a core developer who loves this proposal and accepts it. (But probably not.) (3) Look for some fresh feedback, hopefully (from your perspective) less conservative and negative. Try the "Ideas" topic on Discus. Take into account the feedback you have already received. We've told you the sorts of things that the core devs and Steering Council will be looking for. If you're not willing to provide that, you'll get the same response. Python is a mature language, and the core developers and Steering Council are conservative when it comes to adding marginally useful features that aren't obvious Big Wins, and this is not a Big Win, it is at best a Small Win. This post from one of the core devs may help you understand the culture here: https://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python... My advice to you is that if your time and care factor is low, save your energy for battles you *really* care about. We all want what is best for Python, we may just disagree on what the balance of pros and cons are. If your care factor is high, and you want to continue gathering support for the proposal with the intention of writing a PEP, then listen to those of us who have written successful PEPs in the past.
It is a sign, but how strong a sign is open for debate. My perspective is that replacement of an item in a tuple is *so simple* that it isn't even worth a utility function unless it is generalised. Just build a new tuple using slicing and concatenation expressions. Not every expression is worth an actual named function. I feel that this proposal is *too small* to bother with a function, let alone adding it to the tuple as a builtin method. But generalising it to all sequences, not just tuples, that interests me more. I've used this on strings and lists, probably more often than on tuples. Yes, it raises the bar for acceptance, but it also raises the benefit.
It isn't clear to me what other questions you are referring to. -- Steve
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
Check again in case it's an error. https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp I get 500+ results for each query, including code from several prominent repositories and organizations, such as: - pytorch - pandas - apache - numpy - jax - sympy - cupy - tensorflow - pyqtgraph - aws - dask - deepmind - h5py - cloudflare - facebook research and so on.
My perspective is that replacement of an item in a tuple is so simple that it isn't even worth a utility function unless it is generalised.
I don't oppose the latter.
Just build a new tuple using slicing and concatenation expressions. Not every expression is worth an actual named function. I feel that this proposal is too small to bother with a function, let alone adding it to the tuple as a builtin method.
See Christopher's comments on performance as a builtin. ------- Original Message ------- On Monday, March 14th, 2022 at 7:51 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Mar 15, 2022 at 10:51:21AM +1100, Steven D'Aprano wrote:
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
I tried again, with both URLs, and got plenty of hits this time. Perhaps I had a copy-paste error with the URLs, or a transient error on the website. Those results are promising, but there is a lot of obviously duplicated code there. That's why somebody needs to curate it. -- Steve
On Mon, Mar 14, 2022, 7:52 PM Steven D'Aprano
I got "No results matched your query" on the second URL, at which point I didn't bother with the first.
Hmm... while I don't support the proposal, I saw results at both links. Those links were the first good faith comments I saw from OP. A pretty large number of prominent projects use either: foo = bar[:n] + newval + bar[n+1:] Or: newfoo = list(foo) newfoo[n] = newval foo = tuple(newfoo) Somewhere in their codebases. I find that mild evidence in favor of the idea. But the pattern seems to be "occasionally in large-ish projects." On the other hand, looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?" and 25% "this is easy enough as is." None of them made me think "I wish I had a method/function."
Those links were the first good faith comments I saw from OP.
What do you mean by that, exactly?
looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?"
Give several examples. ------- Original Message ------- On Monday, March 14th, 2022 at 11:08 PM, David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On Mon, Mar 14, 2022 at 11:08:51PM -0400, David Mertz, Ph.D. wrote:
On the other hand, looking through those examples in a cursory manner made me think about 75% of the time "why are they doing this silly approach?"
https://fs.blog/chestertons-fence/ Describing this as "silly" seems, well, silly. What else are you to do when you have a record (a, b, c, d) and need a record (a, b, x, d)? There are only two approaches: mutate in place or create a new record. Which one you do depends on whether you have a mutable or immutable record, and which of those you have depends on parts of the code you can't see in your cursory glance. -- Steve
On 11/03/2022 02:19, wfdc wrote: proposed method.
A function that replaced tuple elements by value rather than by index would no doubt be useful to somebody (not that I'm proposing it). So it could use the same API as str.replace().
Hm, off the top of my head, I could (I believe) write "count" as an (inefficient) 1-liner, but not "index". I suggest it's harder than you think. (Try it!)
"Not every 1-line function needs to be a built-in".
Well, you are 1 user. Have you evidence that there are (many) others? Best wishes Rob Cliffe
But *humans* can be confused by "replace" having a totally different API in different contexts.
I doubt that's the case here. The closest equivalent to tuple's .replace method would be namedtuple's _.replace method, which also has a different API from string's .replace method.
I could (I believe) write "count" as an (inefficient) 1-liner, but not "index". I suggest it's harder than you think. (Try it!)
How much harder? Can you post your candidate? In any case, my point still stands.
"Not every 1-line function needs to be a built-in".
Not every 1-line function needs to *not* be a built-in.
Well, you are 1 user. Have you evidence that there are (many) others? See the StackOverflow link and the 2 other participants in this thread who attested to frequent use of this functionality.
------- Original Message ------- On Friday, March 11th, 2022 at 6:30 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
On Sat, 12 Mar 2022 at 06:33, wfdc via Python-ideas <python-ideas@python.org> wrote:
You still haven't shown why a namedtuple is wrong for your use-case. In fact, you haven't shown anything of your use-case, other than that you've written a one-liner and wish that it were a method. What is the larger context in which this is such an incredibly common operation? In fact, if it's really such a frequent need, maybe you and/or other participants can show more than one use-case. That would be helpful in understanding why tuples need this as a method. ChrisA
You still haven't shown why a namedtuple is wrong for your use-case.
See Christopher Barker's previous reply to you. Furthermore, namedtuples have *fieldnames*. Tuples have *indices*. Tuples are conceptually more appropriate if we're dealing with what are supposed to be numeric indices into some sequential datastructure, rather than a dictionary-like one.
In fact, you haven't shown anything of your use-case, other than that you've written a one-liner and wish that it were a method. What is the larger context in which this is such an incredibly common operation?
1. As already pointed out in the thread, the one-liner is not the most efficient way to implement it, nor does it do bounds checking. 2. See the StackOverflow link and the 2 other participants in this thread attesting to frequent use of this functionality. ------- Original Message ------- On Friday, March 11th, 2022 at 2:36 PM, Chris Angelico <rosuav@gmail.com> wrote:
you haven't shown us what your use-case actually is
Any use-case where you'd want to modify an entry of an immutable sequence. Modifying an immutable datastructure is not a contradictory statement. In fact, there's a whole literature on it. See https://en.wikipedia.org/wiki/Purely_functional_data_structure https://en.wikipedia.org/wiki/Persistent_data_structure As Marco Sulla pointed, out, "Performance apart, this will allow you to code in functional style more easily." namedtuple's ._replace method is an existing example (as is record updating in Haskell, which is a purely functional language, with the attending benefits of immutability). For me *specifically*, my latest use-case is in a game-theoretic setting where the tuple represents a strategy profile. But the details of that would take us far beyond the scope of the discussion.
whether it would actually be more appropriate for a list instead
Lists are not immutable, so they fail the criteria.
Do all your tuples have the same length?
In an inner context, yes. But in an outer context, no. What I mean by that is that, in my particular case, the number of entries is a hyperparameter. ------- Original Message ------- On Friday, March 11th, 2022 at 2:56 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 12 Mar 2022 at 07:38, wfdc <wfdc@protonmail.com> wrote:
you haven't shown us what your use-case actually is
Any use-case where you'd want to modify an entry of an immutable sequence.
That doesn't answer the question any more than your original code does. You're still starting with the assumption that you want to use a tuple, and using that as proof that you need to use a tuple. Show us your ACTUAL use case, your actual code, and why you can't use either a list or a namedtuple. Stop assuming that you are correct before you begin, because *we don't agree* that a tuple is necessary. You have not shown us any code, only made this repeated assertion and then wondered why we don't believe you.
Yes, and dataclasses and namedtuples support this concept very nicely. You still haven't shown why neither works.
I don't understand how a strategy profile has to be a tuple. In fact, particularly since you're wanting to replace one part of it, it definitely seems like a good use-case for a dataclass.
whether it would actually be more appropriate for a list instead
Lists are not immutable, so they fail the criteria.
Why? Do you see how you are still failing to show any actual code, and thus any actual demonstration of why a tuple is necessary? Repeatedly stating that it has to be immutable, has to not be a namedtuple, has to not be a dataclass, and has to have a replace method, doesn't make it true. ChrisA
See https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp and https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp
Yes, and dataclasses and namedtuples support this concept very nicely. You still haven't shown why neither works.
1. They're less efficient, as Christopher pointed out long ago. 2. They're not sequences, so conceptually the wrong tool for the job.
it definitely seems like a good use-case for a dataclass.
See above.
Repeatedly stating that it has to be immutable, has to not be a namedtuple, has to not be a dataclass, and has to have a replace method, doesn't make it true.
See above. ------- Original Message ------- On Monday, March 14th, 2022 at 8:30 AM, Chris Angelico <rosuav@gmail.com> wrote:
A namedtuple IS a sequence. It is a tuple. They are deliberately designed to be drop-in replacements for tuples.
I meant that they're not meant to be addressed as sequences. That's the whole point of namedtuple: to avoid doing that. And the first point still applies. ------- Original Message ------- On Monday, March 14th, 2022 at 12:19 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, 15 Mar 2022 at 04:02, wfdc <wfdc@protonmail.com> wrote:
A namedtuple IS a sequence. It is a tuple. They are deliberately designed to be drop-in replacements for tuples.
I meant that they're not meant to be addressed as sequences. That's the whole point of namedtuple: to avoid doing that.
The whole point is that they are both sequences and structures. https://docs.python.org/3/library/collections.html#collections.namedtuple ChrisA
The whole *point* is that namedtuples let you use *named fields* rather than indices. That's the point. That's the purpose. What are the named fields supposed to be if the datastructure is being treated as a sequence? Indices again? Then what's the point of using namedtuple rather than tuple in the first place? Again, I direct you to Christopher's comment on the matter. ------- Original Message ------- On Monday, March 14th, 2022 at 1:07 PM, Chris Angelico <rosuav@gmail.com> wrote:
On 15/03/22 6:14 am, wfdc via Python-ideas wrote:
The whole *point* is that namedtuples let you use *named fields* rather than indices. That's the point. That's the purpose.
I would say the point of namedtuple is to let you use either field names or indices, whichever makes the most sense for what you're doing at the time. Otherwise why not just use a regular class? -- Greg
On Mon, 14 Mar 2022 at 22:21, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
As I understood it, one motivation for the introduction of namedtuple was that where you had an API that returned a tuple you could change that to return a namedtuple without breaking backwards compatibility. Then older code expecting a tuple could still use tuple-ish idioms like unpacking, slicing or indexing but newer code could use names to access the elements. -- Oscar
See my previous response to Chris:
What are the named fields supposed to be if the datastructure is being treated as a sequence? Indices again? Then what's the point of using namedtuple rather than tuple in the first place?
And, by the way, field names must be valid identifiers.
So again, what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length?
------- Original Message ------- On Monday, March 14th, 2022 at 6:19 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On Tue, 15 Mar 2022 at 10:02, wfdc via Python-ideas <python-ideas@python.org> wrote:
It's very hard to answer a question that is fundamentally based on a misunderstanding of namedtuples. Since a namedtuple *is* supposed to be both a sequence and a record, the names are part of the record interface, and the sequence is still part of that. Also, the field names don't have to be valid identifiers (although they usually will). But if it's a sequence of arbitrary length, then why is it a tuple at all? You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple. Do you see why we've been asking you to SHOW YOUR CODE? I am done. I'm not going to further respond in this thread until there is some actual code to discuss. ChrisA
Again, you neglected to answer the question:
what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length?
Since a namedtuple is supposed to be both a sequence and a record, the names are part of the record interface, and the sequence is still part of that.
The point is that I *don't* want to treat it as a record, and I have no reason to. That's precisely the point I'm making!
the field names don't have to be valid identifiers
Yes, they do.
if it's a sequence of arbitrary length, then why is it a tuple at all?
Why wouldn't it be? A tuple is a sequence. And a sequence can be of arbitrary length.
You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple.
See my first question.
Do you see why we've been asking you to SHOW YOUR CODE?
I already provided *plenty* of examples through the Sourcegraph queries. There's no reason for me to give you my specific code, nor does it help this discussion in any way, since I already gave you plenty of examples. ------- Original Message ------- On Monday, March 14th, 2022 at 7:04 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Mar 15, 2022 at 10:04:45AM +1100, Chris Angelico wrote:
But if it's a sequence of arbitrary length, then why is it a tuple at all?
def func(p, q, *sequence_of_arbitrary_length): ... I guess that the interpreter is Doing It All Wrong and making an XY Problem mistake. Right?
You assume that it has to be a tuple but not a namedtuple, but a namedtuple IS a tuple.
To be precise, namedtuples are a subclass of tuple. For many purposes, we can treat namedtuples and tuples the same way. But namedtuples **extend the tuple API**. In terms of the Liskov Substitution Principle, we can say that any place you use a tuple, you can replace it with a namedtuple, but not the other way around. However, in *practical* terms, the LSP is not the full truth. The problem here is easily seen by considering the *args tuple from function definitions again. If we choose to replace the plain tuple with a namedtuple, what meaningful names would you use for the fields? Aside from the practical problem that namedtuple() doesn't support arbitrary length records, there is no meaningful names for the fields. The best we can do is some mapping of index to ordinals: zeroeth, first, second, third, fourth, fifth, ... but that's hardly meaningful in the sense intended. So namedtuple is a red herring here and the LSP is not the whole truth. Despite what Liskov says, plain vanilla tuples can be considered a superset of named tuples, not a subset: - tuples support records of arbitrary length; - tuples support anonymous fields. The bottom line here is this: There is nothing wrong with using plain tuples as your data structure, and the instance from certain people that wfdc is wrong to do so is arrogant, rude and condescending. -- Steve
And, by the way, field names must be valid identifiers. So again, what are the field names supposed to be if the datastructure is being treated as a sequence, of possibly arbitrary length? ------- Original Message ------- On Monday, March 14th, 2022 at 1:07 PM, Chris Angelico <rosuav@gmail.com> wrote:
How about something like def index(self, x): return next(i for i, a in enumerate(self) if a == x) Including start and end: def index(self, x, start=0, end=-1): return next(i for i, a in tuple(enumerate(self))[start:end] if a == x) ------- Original Message ------- On Friday, March 11th, 2022 at 4:02 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
Please. If we're now talking about specific exceptions, my replace isn't a one-liner either. I was clearly referring to actual functionality. If you want that specific exception, 2 lines are sufficient. ------- Original Message ------- On Friday, March 11th, 2022 at 4:45 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
On Fri, Mar 11, 2022 at 01:38:57AM +0000, Rob Cliffe via Python-ideas wrote:
This could cause confusion because str.replace() has a completely different API.
The old "painter.draw()" versus "gun_slinger.draw()" problem. It does exist, but it's not generally a *major* problem.
A reasonable suggestion. Does replace change items according to their position, or their value? Perhaps a better name would help. wfdc (should we call you something which is not a random-looking string of consonants?) it might help if you could give examples of this functionality from other languages or classes. What do they call the method? It would be nice to have a better API for this function. I have used it many times, for strings, lists and tuples, it is really a generate sequence operation: * make a copy of the sequence, replacing the item at position p with a new value; We can do it with type-specific expressions: seq[:p] + [value] + seq[p+1:] seq[:p] + (value,) + seq[p+1:] string[:p] + "c" + string[p+1:] but there is no good, type-agnostic way to easily do it. Wacky thought: what if index assignment was an expression? function(arg, seq[p]=value, another_arg) would pass a copy of seq with the item at p replaced by value. That would work with slices as well: seq[a:b] = values Now that I have seen this syntax, I want it!!! (It is not clear to me if this would be compatible to the existing `__setitem__ API.)
Whereas your suggestion can be written as a simple 1-liner, as you demonstrate. So there is no strong need for a new method for it.
Sure, not all one-liners need to be built-in. But some do. -- Steve
wfdc (should we call you something which is not a random-looking string of consonants?)
wfdc is fine.
it might help if you could give examples of this functionality from other languages or classes. What do they call the method?
https://clojuredocs.org/clojure.core/assoc https://immutable-js.com/docs/v4.0.0/set()/ http://swannodette.github.io/mori/#assoc https://hackage.haskell.org/package/lens-5.1/docs/Control-Lens-Operators.htm... https://hackage.haskell.org/package/containers-0.6.5.1/docs/Data-Sequence.ht... What about 'update' or 'set'?
what if index assignment was an expression? [...] Now that I have seen this syntax, I want it!!!
Interesting idea. Perhaps we can discuss it further. ------- Original Message ------- On Monday, March 14th, 2022 at 8:38 AM, Steven D'Aprano <steve@pearwood.info> wrote:
jax is another example that uses the name 'set' for this functionality: https://jax.readthedocs.io/en/latest/jax.numpy.html On second thought, perhaps it wouldn't be a bad idea to extend this proposal to built-in sequence types in general (list, str, tuple, and bytes). Examples of this functionality being used on those other types: https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B.*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp If extended beyond tuples, we would have to address the name collision with the existing 'replace' methods of str and bytes. In that case, something like 'set' or 'update' might work. (I prefer the former for its conciseness.) ------- Original Message ------- On Monday, March 14th, 2022 at 12:47 PM, wfdc <wfdc@protonmail.com> wrote:
On Mon, Mar 14, 2022 at 11:16 AM wfdc via Python-ideas < python-ideas@python.org> wrote:
jax is another example that uses the name 'set' for this functionality:
set() strikes me as a better name for a mutating operation. Apparently Jax arrays are immutable, but from the Jax docs: "For example, instead of in-place array updates (x[i] = y), JAX provides an alternative pure indexed update function x.at[i].set(y)" I'd need to read more before I understood that that means (what does x.at[] return? But this does look pretty different than the proposal. Personally, I think the symmetry between this and datatime.replace and namedtuple._replace is actually a good thing, even though the API is a little different. On second thought, perhaps it wouldn't be a bad idea to extend this
proposal to built-in sequence types in general (list, str, tuple, and bytes).
I don't think it should be added to mutable types -- there are already ways to do that, and there are any number of "make a new one with some change" methods that mutables don't have. Essentially, the API of mutable and immutable types is already different. Are you proposing that this be added to an ABC? That would be a heavier lift, and now that I think about it, there is no ABC for immutable sequences -- they are simply sequences -- which don't have any mutating methods. If you are proposing that it be added to a handful of built-in types -- I don't think that should be done for the sake of symmetry, but only if it's useful for that type. -- do we often want to replace the nth character in a string? or the nth byte in a bytes object? Maybe? I don't think I"ve ever needed to do that.
yup -- so I suppose another name would be helpful for future extending. (dict has update() -- would that add confusion?) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Mon, Mar 14, 2022 at 11:53:37AM -0700, Christopher Barker wrote:
How do we make a new list with a change other than the same slice and concatenation we use with tuples? alist[:i] + [value] + alist[i+1:] I mean as an expression, of course we can split it over two statements: newlist = alist[:] newlist[i] = value which is fine, but it does lose the expressiveness of an expression :-) -- Steve
See the following Sourcegraph query for examples of use of this pattern: https://sourcegraph.com/search?q=context:global+lang:python+%5C%5B%5Cs*:%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%5D%5Cs*%5C%2B%5Cs*%5C%28.*%5C%2C%5C%29%5Cs*%5C%2B%5Cs*%5B%5Cw%5C.%5D%2B%5C%5B%5Cs*%28i%7Cj%7Ck%7Cind%7Cindex%29%5Cs*%5C%2B%5Cs*1%5Cs*:%5Cs*%5C%5D&patternType=regexp Regex: \[\s*:\s*(i|j|k|ind|index)\s*\]\s*\+\s*\(.*\,\)\s*\+\s*[\w\.]+\[\s*(i|j|k|ind|index)\s*\+\s*1\s*:\s*\] ------- Original Message ------- On Wednesday, March 9th, 2022 at 10:42 PM, wfdc <wfdc@protonmail.com> wrote:
See also the following Sourcegraph query for examples of use of the second approach: https://sourcegraph.com/search?q=context:global+lang:python+%3D%5Cs*list%5C%28%5B%5Cw%5C.%5D%2B%5C%29%5Cs*%5Cw%2B%5C%5B%5Cw%2B%5C%5D%5Cs*%3D%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%5B%5Cw%5C.%5D%2B%5Cs*%3D%5Cs*tuple%5C%28%5Cw%2B%5C%29&patternType=regexp Regex: =\s*list\([\w\.]+\)\s*\w+\[\w+\]\s*=\s*[\w\.]+\s*[\w\.]+\s*=\s*tuple\(\w+\) ------- Original Message ------- On Friday, March 11th, 2022 at 7:36 PM, wfdc <wfdc@protonmail.com> wrote:
On 2022-03-09 19:42, wfdc via Python-ideas wrote:
Add a "replace" method to tuples that returnsa new tuple with the element at a given index replaced with a given value.
Hmm, no one appears to have mentioned DateTime.replace(). Did I miss it? https://docs.python.org/3/library/datetime.html#datetime.date.replace I think it demonstrates that this is a valid and useful pattern. Agree that I don't often have a need, but am glad it exists. -Mike
participants (12)
-
Brendan Barnwell -
Chris Angelico -
Christopher Barker -
David Mertz, Ph.D. -
Greg Ewing -
Jeremiah Vivian -
Mike Miller -
Oscar Benjamin -
Paul Moore -
Rob Cliffe -
Steven D'Aprano -
wfdc