Operator for inserting an element into a list
I think it would be logical to have the insert operator for lists. Similar to list extend operator += , it could use one of augmented assignment operators, e,g, /=. L = ["aa"] L[0] /= "bb" -> ["bb", "aa"] L[0] /= [1,2] -> [[1,2], "aa"] etc. Without index it would work like append(): L /= "bb" #-> ["aa", "bb"] As for possible spellings I like this one as well: L[i] ^= e The proposed solution is meant to have insert() method semantics, plus it would cover append() method nicely. Insert and append are very frequent operations, so I wonder if there was already related suggestion? Is there some technical problem with implementing this? Note that there is a trick to 'insert' an element with slicing syntax, e.g.: L[0:0] = [[1,2]] -> [[1,2], "aa"] L[0:0] = ["bb"] -> ["bb", "aa"] The trick is to put brackets around the element and so it works as insert(). Though additional brackets look really confusing for this purpose, so I don't feel like using this seriously. M
That's a slice assignment, works great. I think if you use it more often you'll start to enjoy it. However, lists are optimized for append. Insert is slow. It should be discouraged, not encouraged by the language. If inserting is just a tad more awkward than appending, that's the language design giving you a hint that you should do what reads beautifully. On Tue, Jun 12, 2018, 7:54 AM Mikhail V <mikhailwas@gmail.com> wrote:
On Tue, Jun 12, 2018 at 5:54 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Oops Sorry for confusion, I inserted wrong examples here: The examples should be : L[0:0] /= "bb" L[0:0] /= [1,2] ... L[i:i] ^= e Of course. Because L[i] /= is an operation on list element, thus already working syntax.
-1. There's not much about this that is logical, no matter how much you want an insertion operator. Even if L[0] /= "bb" worked, then logically so should L[0] = L[0] / "bb". However, there is no sense in which L[0] / "bb" by itself has any meaning, and what would L[1] = L[0] / "bb" mean? And finally, L[0] /= x (and really, every other augmented operator) *already has* a meaning:
It's no more confusing than co-opting an unrelated operator to do the same thing. -- Clint
On Tue, Jun 12, 2018 at 7:42 PM, Clint Hepner <clint.hepner@gmail.com> wrote:
Hi Clint, (and others), I must say it is misunderstanding due to my false examples, that I have pasted in original post (I had 2 cloned texts in my text editor and copied the wrong one). I have posted correction just 10 minutes after the original post, see last post. Sorry for confusion! So the idea was about an insert/append operator. Which would use augmented operator. The operator may be /= or ^=. (I like ^= more, so I'll put here example with it). L = [1,2,3] L[0:0] ^= 0 -> [0,1,2,3] L[0:0] ^= -1 -> [-1, 0, 1, 2, 3] L ^= 4 (without index, it works as append() ) -> [-1, 0, 1, 2, 3, 4] As for your question, what would: List1[a:b] = List1[c:d] ^ var mean? (is that what you have asked?) Well, I think this would mean simply : - first append var to List1[c:d] , - then replace the List1[a:b] part with the result. So at least L ^= 4 would make sense as L = L ^ 4. Actually current semantics of += for lists: L += var and L = L + var are different, so it seems to me they were not meant to be bound together.
If the intention is to insert an element, indeed it is confusing, for me at least: L[0:0] = "bb" -> ["b","b","aa"]
On Tue, Jun 12, 2018 at 11:08 AM Mikhail V <mikhailwas@gmail.com> wrote:
The "/" operator means divide. How do you divide a list? The "^" operator means exclusive-or. Again, strange concept for a list. Yes, there are examples of Python (mis)using operators like "%" for string interpolation. Note the difficulties that caused; we now have 3 ways to interpolate. It looked cool at the time, but happily we have f-strings now.
One is mutation, the other isn't, but aside from that, the result is equivalent.
Thats, like, just your opinion, Man. Kidding aside, I think you'll find it's natural once you get used to slice assignment. The slice assignment iterates over the right-hand argument and inserts each element. It overwrites the existing slice if any. In this example, you've mixed issues. It looks like you've gotten confused by string iteration. When you loop a string, you get 1-len strings.
On Wed, Jun 13, 2018 at 5:25 AM, Michael Selik <mike@selik.org> wrote:
I agree about XORing a list, but dividing a list could conceivably be implemented to split a list into parts. For instance: low, high = list(range(10)) / 2 But it wouldn't mean "insert". Also, I can't imagine an augmented division operator being useful, but others may disagree. ChrisA
On Tue, Jun 12, 2018 at 10:25 PM, Michael Selik <mike@selik.org> wrote:
Allowed standard characters are so - they already mean something and there are so few (even less if throw away ugly looking). For me the plus character + means sum of numbers. So for an array: A += 1 Means for me unequivocally increment each element of array by 1. And this: A += B Means for me increment each element of A by corresponding values of B. BTW that is how plus operator works for Numpy arrays. So I don't think it is a precise thing how logical this or that character suits an operation. Everybody understands that overloading of operators is for convenience and it's just a shortcut for some frequent usage. As I understand, overloading some operators for certain object type (list in this case) has relatively low cost in terms of implementation. As for me - I'm fine with append() method and use it all the time. Frankly speaking, I was working on another syntax idea and it turned out that actually augmented operators may be very useful for that particular syntax design.
You're right of course. I was just confused by the fact that L += "aa" # works L = L + "aa" # gives TypeError But for consistent types both should work the same.
I think you'll find it's natural once you get used to slice assignment.
I use slice assignment all the time with Numpy arrays, though for list element appending I prefer append() method. M
On 6/12/2018 10:54 AM, Mikhail V wrote:
...
Note that there is a trick to 'insert' an element with slicing syntax, e.g.:
This is not a 'trick'. It is a particular case of a general operation: replacing a length m slice of a list with a sequence of length n. Both m and n can be 0. The replacement sequence can be any iterable.
In these examples, m and n are 0 and 1.
The trick is to put brackets around the element and so it works as insert().
Again, not a trick. Putting brackets around the element makes it sequence of length 1. To possible be less confusing, you could use (,)
Though additional brackets look really confusing for this purpose, so I don't feel like using this seriously.
Learning about lists means learning about slice assignment: replace a sublist with another sequence. -- Terry Jan Reedy
On Tue, Jun 12, 2018 at 10:26 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Yes I see, actually that is what I am saying - slice assignment has _replace_ semantics. If that would be a list method, I suppose it would work something like: L = L.replace(begin, end, L2) So that makes perfect sense. But returning to append/insert. So appending of course is extremely frequent operation, inserting in the beginning of the list may be also not rare. Concatenation in my opinion might be not so frequent. Writing values from one array (or slice) to another is quite frequent operation, that means slices of the same size. As for _replacing_ slices, of different size, I personally never used it. OTOH it smells like generalization. Anyway, I prefer to look at it through 'syntax glass'. Here is something ubiquitous: L = L.append("string1") L = L.append("string2") L = L.append("string3") L = L.append([1,2,3]) Could be written: L ^= "string1" L ^= "string2" L ^= "string3" L ^= [1,2,3] These kind of things are basically in every second python script. I'm ok with append() actually, but it just asks for a shortcut. For the ^ character - yes it looks strange. And writing it like this: L += ["string1"] L += ["string2"] L += ["string3"] L += [[1,2,3]] Sorry, no, that would be too much for me, also this needs some real mental training so as not to misinterpret these. Append() is just fine, though it's a pity there is no shortcut operator. M
On Tue, Jun 12, 2018 at 4:43 PM Mikhail V <mikhailwas@gmail.com> wrote:
inserting in the beginning of the list may be also not rare.
Inserting in the beginning of a list is *slow* and should be rare. If you want to append to the left-side of a list, you should use a deque. Check out ``collections.deque`` for your insert(0, x) needs.
^ is also used in regexes for matching the *beginning* of a string... Realistically, I don't think this proposal would be added, but if it were, ^ would be a horrible choice. That being said, I do understand the feeling of half your code being calls to .append or .extend. You could always do: L += 'string1' Only thing extra is the comma at the end; it converts the single value to a tuple, which is then added to the list. As for the use of +, I think that's mostly opinion. Many other languages use + for concatenation, so Python's hardly on its own here. On June 12, 2018 6:43:18 PM Mikhail V <mikhailwas@gmail.com> wrote:
Mikhail V wrote:
I don't think it's all that confusing. It looks a bit cluttered when the thing being inserted is itself a list literal, but that seems like a rare case of something that's not all that coommon in the first place. My feeling is that inserting is not a frequent enough operation to warrant having its own operator, especially not when there is already a syntax that does the same thing.
Is there some technical problem with implementing this?
Yes, it doesn't fit into the current scheme for augmented assignment operators. There are no special methods for combining augmented assignments with slicing -- you couldn't implement this just by adding an __ixor__ method to the list type. There would need to be a new special method for "in-place xor with slice", and the compiler would have to recognise this combination and emit special bytecode for it. That would raise the question of why ^= is getting this special treatment but not any of the other augmented assignments, and why not "in-place operation with attribute" as well, and potentially we would end up with two new entire sets of special methods for different flavours of augmented assignments. I really don't think we want to go there. -- Greg
On Wed, Jun 13, 2018 at 2:15 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Mikhail V wrote:
Depends on what you count as 'insert' - append is one case of insert ;) (logically seen) Sorry for repeating myself, the idea was that the default meaning is append(), i.e. normal operator usage on list: L1 = L2 ^ item - would be same as L1 = L2.append(item) But hope you get the point - if an operator for append was added, then it would be a bit sad that it cannot be used for inserting by slicing, namely these two forms: L ^= item #append(item) L[i:j] ^= item #insert(i, item) instead of i:j items would be IMO nice to have and it 'd cover both insert and append. But if you say that special-casing of [i:j] here would be hard to implement, then maybe insert() idea should be dropped.
As said, I don't insist on ^ operator. I would find ">>" or "<<" or "|" ok as well: L <<= "foo" L1 = L2 << "foo" L |= "foo" L1 = L2 | "foo" But it would raise same questions. As for relation 'by sense', that may be too opinion based. Vertical bar "|" may be somewhat related - IIRC in some contexts it is used as element separator. Just hope the judgement is not like "the symbol looks strange - therefore the feature is not needed". M
On Thu, Jun 14, 2018 at 12:04 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Not sure exactly what your intention here is, because list.append mutates the list and returns None. Does "L2 ^ item" mutate L2 in place, or does it construct a new list? If it mutates in place, does it return the same list? Or if doesn't, how is it different from "L2 + [item]", which is a much more logical spelling of list addition? ChrisA
On Thu, Jun 14, 2018 at 12:40 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Okay. Now it all is coherent and makes perfect sense... but you're offering alternative spellings for what we can already do. The only improvement compared to the + operator is that you don't need to surround the operand in brackets; in return, it's less general, being unable to add multiple elements to the list. The only improvement compared to .append is that it's an operator. There's no connection to exclusive-or, and there's not a lot of "but it's intuitive" here (cf Path division). ChrisA
On Wed, Jun 13, 2018 at 5:46 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Jun 14, 2018 at 12:40 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Exactly, IMO it's important to make distinction for the item append. Not only for practical reason, but for semantical distinction, just like append() vs extend() distinction. Otherwise if += is used as append(), it creates an illusion that += _is_ append().
Mikhail V wrote:
Okay, that achieves an in-place append, but it's not exactly obvious to the unenlightened what it does, whereas append() is pretty self-explanatory. Also, using the slice version to do an insert L[i:i] ^= item is not as efficient as it looks like it should be, because it creates an empty list, appends the item to it and then splices that back into the list. And you have to write the index twice. Whereas L.insert(i, item) doesn't have any of those problems, and again is mostly self-explanatory. Python is not Perl. Not every operation that you use more than once in a blue moon needs to have its own operator. -- Greg
On Thu, Jun 14, 2018 at 2:58 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Sure, I use append() only and obviously would recommend to do so. I am not sure though everything's so simple here. Yet by writing examples here I've made typos several times, like L = L.append() - so strong is the influence of functional programming. I don't like cryptic operators, although such cases make me think in-place operations need some specialty in their syntax and augmented assignment could provide something positive in this regard. Another point is that people do like augmented operators much and for the append - there are so many advises like: hey, use L += [item] ! But Imo this just makes things worse from both practical and semantics POV (still I find it hard to explain why exactly, that's just some gut feeling). Not to argue, since I personally wouldn't benefit much from the idea, but it just seems important, more than just 'cosmetics'.
Efficiency is important ... where it is important.
On Wed, Jun 13, 2018 at 6:45 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Another point is that people do like augmented operators much and for the append - there are so many advises like: hey, use L += [item] !
another data point -- in teaching, a number of newbie students do exactly that. Actually, they do: a_list += an_item and find it does not do what they want, and then the get confused, and I show that they need: a_list += [an_item] or a_list.append(an_item) (this gets particularly confusing them an_item is, itself, a sequence (a string is really common). So it would be nice to have an operator version of append -- but given the limited number of operators, and their usual uses, I suspect it would cause even more confusion... But throwing it out there, how about (ab)using the mat_mul operator: a_list @= an_item Another note: One of the major motivations for augmented assignment was being able to support in-place operations for numpy: an_array += something which was MUCH nicer notation that was was required: np.sum(an_array, something, out=an_array) That is a much bigger win than going from .append() to an operator. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Jun 14, 2018 at 7:40 PM, Chris Barker <chris.barker@noaa.gov> wrote:
On Wed, Jun 13, 2018 at 6:45 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Yes, none of the oprators fits well by sense. But which symbol would fit here if you could choose *any* symbol? I've picked the caret ^ for two reasons: 1. It is originally associated with the insertion mark.
So there is some relation with "insert" operation. But of course this makes sense mainly for people who have some editorial or typography background, (such as myself). So for me it makes perfect sense. 2. The symbol ^ itself looks quite 'gentle' and to my eye, is not too distracting (at least less than some other operators). So e.g. I find that vertical bar | causes some bad eye straining effect.
Agreed.
There's nothing wrong with your ideas if you were designing a language from scratch. However, Python has a long history and many tools and uses for the same operators you are considering. And even has a current "insert" operator (slice assignment). When adding a new feature, you need to consider if you're duplicating current functionality and if a 2nd "obvious" way is worth breaking the Zen. Also, consider whether your "intuitive" design is following or breaking a de-facto standard established elsewhere in the language. For example, if you're making a file format decoder, you should create functions "load" and "dump" to behave like every other file format decoder module. Don't re-invent or we'll end up with too many "standards" and the language will become hard to remember. In case I need to clarify: 1. You're duplicating current clear and more flexible syntax. 2. Your proposed operators are confusing when compared with their meanings elsewhere.
On Fri, Jun 15, 2018 at 1:04 AM, Michael Selik <mike@selik.org> wrote: [..]
what haven't we repeated in this thread yet? Motivation was explained. About confusion probably haven't discussed : so Xor ^ works on sets, IIRC for finding union without common elements. That's one point for potential confusion - probably expect it to work with list. Sounds probable? Other operator, e.g. bitshift <<, I dont' think it has potential for confusion. @ operator - currently 'free'. But maybe there is some reserved plans for lists and these operators as well - I don't know. IIRC some time ago Steven D'Aprano proposed something with sets and some of the operators (sorry in advance if that is wrong info, but I\m almost sure there was a related mathematical discussion with sets involved).
On Thu, Jun 14, 2018, 7:24 PM Mikhail V <mikhailwas@gmail.com> wrote:
what haven't we repeated in this thread yet? Motivation was explained.
You have repeated your explanations a few times. It isn't convincing. It seems to me that your main complaint is that strings are iterable, though you haven't expressed it as such. During slice assignment, you're surprised that the string is a sequence of characters. I understand, but I don't find that confusion a compelling reason to add a new operator. About confusion probably haven't discussed : so Xor ^ works on sets,
IIRC for finding union without common elements.
I wrote about this in an earlier email, which you didn't reply to. You're correct that xor doesn't make sense with lists, especially not between a list and a single element. Again, though other operators have been given different meanings in special contexts, those decisions should be seen as abnormal, not an excuse to keep creating more special cases.
Sorry, I forgot that you dropped the suggestion to make it an insert operator and are only asking for an append operator. I see no benefit to this, because += already is an elegant way to extend a list, which is more flexible than append. Yes, if the right-hand is an iterable and should be appended as a single element, you'll need to enclose it in a single-element container. This is true for strings, lists, sets, whatever. It's natural and is not a "trick". If you would like to prove the need for this operator, one piece of evidence you can provide is a count of the number of times someone writes "list.append" for an iterable vs "+=" and encloses a str or other type in a throw-away list to effectively append. If the latter habit is common, that's evidence that the language may need to be improved. Why don't you search GitHub projects to collect some statistics? On Thu, Jun 14, 2018, 7:40 PM Michael Selik <mike@selik.org> wrote:
On Fri, Jun 15, 2018 at 5:51 AM, Michael Selik <mike@selik.org> wrote:
That's strange idea - there is no doubt that one would use list.append() and most probably it is the case statistically. So the question would be "what is wrong with list.append()?" And as said many times, there is nothing wrong, but a lot of people seem to want an in-place operator for this purpose. And I can understand this, because: 1. append() is _ubiquitous_ 2. in-place assignment form makes some emphasis on mutating, in contrast to method call. That's it. So instead of a method call one gets a clean element on the right-hand and (hopefully) emphasis on the in-place nature of operation. A quick google search shows some tendency: https://stackoverflow.com/a/2022044/4157407 https://stackoverflow.com/a/28119966/4157407 So you shoudn't explain it to _me_ - I don't see other significant convincing points, and unless this gets support here - I am not interested in continuing.
encouraging to mimic append() via += operator is bad practice. In the above links to SO, people try to tell the same, but that is not something that can be easily explained on paper. Now add here the constant wish for using in-place operator - and there will be on-going confusion. Regardless of how natural or general it is, it's something that should be avoided. Only in this discussion thread I had 3 different advices to use: L += ["aa"] L += ("aa",) L += "aa", But you should not give it to _me_ - i use the correct form only: L.append("aa")
On Sat, Jun 16, 2018 at 1:48 AM, Mikhail V <mikhailwas@gmail.com> wrote:
How so? You can write "x += 1" with integers, and that doesn't mutate; but if you write "x.some_method()", doing nothing with the return value, it's fairly obvious that it's going to have side effects, most likely to mutate the object. Augmented assignment is no better than a method at that. ChrisA
On Fri, Jun 15, 2018 at 6:54 PM, Chris Angelico <rosuav@gmail.com> wrote:
How it would be obvious unless you test it or already have learned by heart that x.some_method() is in-place? For a list variable you might expect it probably, and if you already aware of mutability, etc. It s just very uncommon to see standalone statements like: x.method() for me it came into habit to think that it lacks the left-hand part and =. Of course augmented assignment is not a panacea because it is limited only to one operation, and the appeal of the operator itself is under question. As for x+=1 it is implementation detail - historical idea of such operators was mutating, so at least visually its not like a returning expression. and I am not sure about x.method() form - was it meant to hint to the user about anything? It seemed to me so when I started to learn Python, but its not.
On Fri, Jun 15, 2018 at 10:25 AM Mikhail V <mikhailwas@gmail.com> wrote:
very uncommon to see standalone statements like: x.method()
Python has many such mutation methods. It sounds like you're judging the frequency of code patterns across all languages instead of just Python. Even then, I don't think that's true. All OO languages that come to mind have that pattern frequently. As for x+=1 it is implementation detail - historical idea of such operators
was mutating, so at least visually its not like a returning expression.
Incorrect. The += operator was meant as an alias for ``x = x + 1``. The fact that it mutates a list is somewhat of a surprise. Some other languages make no distinction between mutation and reassignment. Perhaps you're thinking of one of those other languages.
Yes, it returns None to emphasize that it's a mutation. This is different from the so-called "fluent" design pattern where all mutation methods also return the original object, causing confusion about whether the return value is a copy or not.
Michael Selik wrote:
The += operator was meant as an alias for ``x = x + 1``. The fact that it mutates a list is somewhat of a surprise.
That's very much a matter of opinion. For every person who thinks this is a surprise, you can find another that thinks it's obvious that += should mutate a list, and is surprised by the fact that it works on immutable types at all.
Most languages, actually. I'm not aware of any other language with a += operator that has this dual interpretation of its meaning -- which is probably why almost everyone gets surprised by at least one of its meanings. :-) Maybe we should call it the "Spanish Inquisition operator". -- Greg
On Sat, Jun 16, 2018 at 01:06:45PM +1200, Greg Ewing wrote:
Given the ubiquity of += in C, where it works on numbers but not lists, and the general difficulty many people have in dealing with the difference between assignment and mutation, I think the ratio would be closer to 20:1 than 1:1. But regardless, I'm pretty sure that nobody expects this: py> t = ([], None) py> t[0] += [1] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment py> print(t) ([1], None)
surprised by at least one of its meanings. :-)
Maybe we should call it the "Spanish Inquisition operator".
:-) -- Steve
On Sat, Jun 16, 2018 at 9:09 PM, Steven D'Aprano <steve@pearwood.info> wrote:
The nearest you'd get to "adding to a list/array" in C would be adding to a pointer. But I don't see people writing code like this in Python:
:)
Agreed, although I can't remember this ever coming up outside of interactive work. When you do something at the interactive prompt, you can kinda feel that it should "rollback" on exception, and yet some part of it has happened. But in most scripts, that TypeError is going to pull you all the way out of the scope where 't' exists - frequently, it'll just straight-up terminate the script - so you won't often see this. ChrisA
On 6/15/18 1:25 PM, Mikhail V wrote:
For me if I see foo.bar() my assumption is that likely bar() is going to mutate foo, especially if it doesn't produce a return value, and this is the natural way to define something that mutates something. If it returns something, it might be just an accessor, or it might be a bit of both, easily a mutation that returns a status about the success/failure of the mutation. foo = bar would seem to be never a 'mutation', but always a rebinding of foo, even if 'bar' is an expression using foo. foo += bar, conceptually reads as foo = foo + bar, so the first impression is that it isn't going to mutate, but rebind, but it is possible that it does an inplace rebind so to me THAT is the case where I need to dig into the rules to see what it really does. This means that at quick glance given: list1 = list list1 += value it is unclear if list would have been changed by the +=, while a statement like list1.append(value) is more clearly an in place mutation so other names bound to the same object are expected to see the changes. -- Richard Damon
Now I have slightly different idea. How is about special-casing of this as a shortcut for append: L[] = item Namely just use the fact that empty slice is SyntaxError now. I understand this is totally different approach than operator overloading and maybe hard to implement, but I feel like it looks really appealing. And it is quite intuitive imo. For me the syntax reads like: "add new empty element and this element will be "item". No?
On Sat, Jun 16, 2018 at 3:02 AM, Chris Angelico <rosuav@gmail.com> wrote:
Is it like that in PHP?
I still haven't seen any compelling argument against the append method. -1 on introducing a new way to spell append.
By me - there is just nothing against append() method. But I see from various posts in the web and SO - people just want to spell it compact, and keep the 'item' part clean of brackets. And that kind of makes sense.
On Fri, Jun 15, 2018, 5:24 PM Mikhail V <mikhailwas@gmail.com> wrote:
there is just nothing against append() method.
Then why break the Zen: there should be only one obvious way? But I see from various posts in the web and SO - people just want to spell
it compact, and keep the 'item' part clean of brackets.
If you're going to cite things, please link to them.
On Sat, Jun 16, 2018 at 3:26 AM, Michael Selik <mike@selik.org> wrote:
I think the question could be applied to 99% proposals
Want to force me to work? :) I made pair of links already in previous posts, not directly telling concrete wishes, but shows approximately the picture. Something more concrete maybe: https://stackoverflow.com/a/3653314/4157407 https://stackoverflow.com/q/13818992/4157407
On Sat, Jun 16, 2018 at 3:47 AM, Michael Selik <mike@selik.org> wrote:
One of those links was discussing extend, not append.
Yes and so what? Does this makes it automatically not related to the wish to choose more compact spelling, despite it is not recommended way. What is different with append? Similar posts are in topics about append and anything else actually, e.g.: i = i +1 i +=1 You can find enough evidence that people prefer i += 1.
The other wanted a repeated append, best solved by a list comprehension.
I think yuo should reread that one more thoroughly - I'll even paste here the text: """ When appending longer statements to a list, I feel append becomes awkward to read [...] Though append works just fine, I feel it is less clear than: mylist[i] = 2*math.pi*radius*math.cos(phi[i]) """" So your claim this ^ is not relevant?! Seriously I am starting to get tired of that style of conversation. I provided you links - you are not pleased again.
On Fri, Jun 15, 2018, 6:18 PM Mikhail V <mikhailwas@gmail.com> wrote:
Luckily for extend, it's similar to the "obvious" semantics of ``a += b`` which is ``a = a + b``. Unfortunately for append, there's nothing quite like it among the operators.
mylist[i] = 2*math.pi*radius*math.cos(phi[i])
So your claim this ^ is not relevant?!
I read that sentence. No, it was not relevant, because the best answer was to teach that person about list comprehensions, not to offer a new syntax. Seriously I am starting to get tired of that style of conversation.
To be honest, I'm getting a little tired myself. I am trying to politely suggest ways to strengthen your proposal even though I disagree with it. I provided you links - you are not pleased again.
Are you aware that modifying the language is difficult, time consuming, and the folks that do it aren't paid for their work? Further, any change is likely to increase the maintenance burden on these same volunteers. Even worse, tens of thousands of teachers will need to add more time to their lesson plans to explain new features. Book authors will need to issue errata and revised versions. If you add that all together, in a sense, changing the parser to expand the syntax would cost millions of dollars. Is that worth the change?
On 16Jun2018 02:42, Mikhail V <mikhailwas@gmail.com> wrote:
Now we're just making typing errors into working code. Also, that isn't an empty slice. That's a _missing_ slice. An empty slice has zero length. While genuinely new syntax needs to land in such a gap (because otherwise it will break working code), new syntax needs a much highly value than new meanings for existing operators. Some things _should_ be syntax errors. Particularly things which may be typing errors. Suppose I'd meant to type: L[0] = item Silent breakage, requiring runtime debugging.
The term "new empty element" is a nonsense term to me. If you mean "replace an empty slice at the end of the list with a new element", that can already be written: L[len(L):len(L)]=[9] Cumbersome, I accept. But I've got a .append method. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, Jun 16, 2018 at 4:44 AM, Cameron Simpson <cs@cskk.id.au> wrote:
On 16Jun2018 02:42, Mikhail V <mikhailwas@gmail.com> wrote:
Not sure that it's different from any other situation: e.g. if by mistake I will write: L[i+1] instead of L[i+2] Visual difference between L[] = L[1] = is big enough to reduce the typing error. But maybe I did not understand your example case. FWIW in general, claims about possible typing errors by introducing this or that syntax are speculative. Main source of typos usually is high similarity of spellings and characters, or initial obscurity of spelling - e.g. excessive punctuation. Does it apply here?
I just say that I find this syntax is more intuitive than the idea with operator. Not sure that we need to start semantics nitpicking. For example, such code: L = [] L[] = x L[] = y imo has more chance to be understood correctly than e.g.: L = [] L ^= x L ^= y By L[] there is some mnemonical hint because [] is used to create new empty list. Plus it does not introduce overloading of the operator. And overloading has weakness in this - e.g. " var1 += var2 " does not have mnemonics, other than + character (it could be two integers as well). So if L[] is found somwhere far from initialisation - it may be a good aid. It makes it more clear what is happening, compared to augmented operator.
On Sat, Jun 16, 2018, 10:22 AM Mikhail V <mikhailwas@gmail.com> wrote:
Plus it does not introduce overloading of the operator.
Now you're critizing duck typing. And overloading has weakness in this - e.g. " var1 += var2 " does not
have mnemonics, other than + character (it could be two integers as well).
That's an advantage, not disadvantage.
On 16Jun2018 20:21, Mikhail V <mikhailwas@gmail.com> wrote:
Well, many typing errors are invalid syntax. When that happens you find out immediately.
This is true: many other typing errors are not syntax errors.
No, you understand the example; we differ in our estimation of likelihoods and costs to mistakes. Particularly, I dislike "silent breakage", which aan be much harder to fix because it shows as incorrect behaviour far from the error (eg counters being slightly wrong leading to misbehaviour in things depending on the counter). Assuming we are lucky enough for the misbehaviour to be obvious. But there is also the point that _every_ new piece of syntax reduces the surface of "invalid syntax that can catch simple mistakes". So new syntax tends to requires a higher perceived benefit than, say, a new feature on a class/type. Such as your suggestions about having lists support more operators i.e. "^" to mediate list insertion.
FWIW in general, claims about possible typing errors by introducing this or that syntax are speculative.
Certainly.
Unsure, depends on the programer.
Well, as someone else pointed out, PHP has this notation for append. It is quite convenient. (In fact, it is one of the very few convenient things in PHP.) However, PHP is generally not a winning source of ideas in the Python world. Certainly I was _very_ glad to not be suffering with PHP once I left my previous job. There were many many things to like about my previous job, but PHP was not one of them. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, Jun 16, 2018 at 08:21:42PM +0300, Mikhail V wrote:
Should be written as L = [x, y].
I disagree. The first syntax L[] = x looks so similar to L[:] assignment that I keep reading it as "set the list L to a single item x". It certainly doesn't look like an append operation. The second at least looks like a mutation on L.
By L[] there is some mnemonical hint because [] is used to create new empty list.
How is that a hint? What is the connection between "append an item" and "create a new empty list"?
So if L[] is found somwhere far from initialisation - it may be a good aid. It makes it more clear what is happening, compared to augmented operator.
I don't think so. Here is a radical thought... why don't we give lists a method that inserts items at the end of the list? We could call it something like "append", and then instead of hoping people guess what the syntax does, they can just look up the name of the method? L.append(x) might work. *wink* -- Steve
On Sun, Jun 17, 2018 at 2:52 AM, Steven D'Aprano <steve@pearwood.info> wrote:
On Sat, Jun 16, 2018 at 08:21:42PM +0300, Mikhail V wrote:
Where did I say it has _direct_ connection? It has some associative connection - 'new item', 'special index case'. L = [] is new list which is supposed to be filled with something. And it has references to existing syntax, e.g. slice assignment, or adding dictionary item can be written as : mydict[key] = value so: mylist[] = item is not THAT far. I would even say it's very close - but I'm pretty sure you can find something against this as well.
Exercising in wit? How about: let's assume most people can't understand even the simplest new feature. So instead of giving clean compact assignment syntax for frequent operation, let's force less readable method call everywhere, which also might be harder to remember than one index case []. *wink* Maybe force .extend() for lists and .update() for dicts everywhere? To preserve poor new users from 'unintuitive' assignment syntax. But how you would do that?
On Sun, Jun 17, 2018 at 02:43:16PM +0300, Mikhail V wrote:
I didn't mention "direct" connection. You said there is a mnemonic from "the empty list" to "L[] is used for insert". I don't know what that mnemonic is, and your description below doesn't make sense to me. Insert/append doesn't just work on empty lists, so the connection between "empty list" and inserting/appending is pretty tenuous. Aside from the presence of a list, I don't see any connection at all. If we think about this example: L = ["starting", "values", "go", "here"] for item in extra_values: L[] = item do you still see a connection between a non-empty list and L[] used for append?
It has some associative connection - 'new item', 'special index case'. L = [] is new list which is supposed to be filled with something.
I don't see any connection between "new item" and "special index case" either.
I would expect mylist[] = value to mean that the contents of mylist is replaced with a single item. And then I would wonder why this is so special that it needs dedicated syntax, when we can already do it with existing syntax: mylist[:] = [item] The idea that assignment to mylist[] means "append to the end of the list" would never cross my mind in a million years.
Compact it might be: L[] = x # seven characters, including spaces L.append(x) # eleven characters but I think that calling it "clean" is inappropriate. Slice notation is already one of the trickier things for beginners to learn, and this adds extra complexity for not much benefit. As for "frequent operation", there are lots of frequent operations in Python. Does every one of them deserve special syntax to make it clean? I just opened one of my modules at random, and I don't have a single append in that module, but I have 14 calls to string.startswith and nine calls to kwargs.pop. Appending to a list might be common, but I don't see that it is either common enough or important enough to add additional syntax.
let's force less readable method call everywhere,
I think that describing a short, self-descriptive method call like append as "less readable" demonstrates a deep and fundamental gulf between the style you prefer and what the rest of us prefer. Mikhail, sometimes I wonder if you would be happier using Perl rather than Python. Like the Perl community, you seem to have a desire for syntax which most of us see as terse and cryptic over self-descriptive method names. But even Perl doesn't (so far as I know) give us special syntax to append to an array. As far as I know, the standard way to append to an array @arr in Perl is: push(@arr, item); -- Steve
Mikhail, this thread is getting quite long, and difficult to follow. It's quite clear that a new operator won't be introduced for list insertion. Furthermore, this thread has now become a hard-to-follow and impossible-to-participate-in debate. If you have other ideas, could you maybe formulate them unambiguously and post them in a new thread? With examples. I'm interested in reading about new ideas, but not rhetorics and argumentations. Best wishes, Pål Grønås Drange
On Sun, Jun 17, 2018 at 11:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
kwargs.pop("some_key") could plausibly be spelled del kwargs["some_key"] if del were (like yield) upgraded to expression. Whether that is an improvement or not, I don't know, but at least it's logical. ChrisA
Mikhail V wrote:
You must be looking at a very limited and non-typical corpus of Python code. Mutating method calls are extremely common in most Python code I've seen. You seem to be trying to reason about Python as though it were intended to be used in a functional style, but it's not. Making guesses about it based on how something would be done in a functinal language will get you nowhere.
I am not sure about x.method() form - was it meant to hint to the user about anything? It seemed to me so when I started to learn Python, but its not.
The fact that something is a method does not, and was never intended to, imply anytbing about whether it is mutating. However, there *is* a fairly easy way to tell, most of the time, when you're *reading* code. There's a convention that mutating methods don't return anything other than None, so mutating method calls reveal themselves by the fact that the result is not used. Conversely, a method call whose result *is* used is most likely non-mutating. (It's possible that it could both return a value and have a side effect, but that's frowned upon, and you'll pretty much never see it in any builtin or stdlib API.) As for writing code, you just have to rely on memory and documentation, just like you do with most aspects of any language and its libraries.
Mikhail V wrote:
But if you say that special-casing of [i:j] here would be hard to implement, then maybe insert() idea should be dropped.
Since I wrote that I realised that it's not true -- given an infix ^ operator like you propose, the in-place version of it would actually give the desired result. However, I still don't think that either insert or append are frequent enough operations to warrant having their own operators. It's true that append is a lot more common than insert, but usually it's a mutating append that you want, not creating a new list with one more item on the end, which is what your ^ operator does. -- Greg
That's a slice assignment, works great. I think if you use it more often you'll start to enjoy it. However, lists are optimized for append. Insert is slow. It should be discouraged, not encouraged by the language. If inserting is just a tad more awkward than appending, that's the language design giving you a hint that you should do what reads beautifully. On Tue, Jun 12, 2018, 7:54 AM Mikhail V <mikhailwas@gmail.com> wrote:
On Tue, Jun 12, 2018 at 5:54 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Oops Sorry for confusion, I inserted wrong examples here: The examples should be : L[0:0] /= "bb" L[0:0] /= [1,2] ... L[i:i] ^= e Of course. Because L[i] /= is an operation on list element, thus already working syntax.
-1. There's not much about this that is logical, no matter how much you want an insertion operator. Even if L[0] /= "bb" worked, then logically so should L[0] = L[0] / "bb". However, there is no sense in which L[0] / "bb" by itself has any meaning, and what would L[1] = L[0] / "bb" mean? And finally, L[0] /= x (and really, every other augmented operator) *already has* a meaning:
It's no more confusing than co-opting an unrelated operator to do the same thing. -- Clint
On Tue, Jun 12, 2018 at 7:42 PM, Clint Hepner <clint.hepner@gmail.com> wrote:
Hi Clint, (and others), I must say it is misunderstanding due to my false examples, that I have pasted in original post (I had 2 cloned texts in my text editor and copied the wrong one). I have posted correction just 10 minutes after the original post, see last post. Sorry for confusion! So the idea was about an insert/append operator. Which would use augmented operator. The operator may be /= or ^=. (I like ^= more, so I'll put here example with it). L = [1,2,3] L[0:0] ^= 0 -> [0,1,2,3] L[0:0] ^= -1 -> [-1, 0, 1, 2, 3] L ^= 4 (without index, it works as append() ) -> [-1, 0, 1, 2, 3, 4] As for your question, what would: List1[a:b] = List1[c:d] ^ var mean? (is that what you have asked?) Well, I think this would mean simply : - first append var to List1[c:d] , - then replace the List1[a:b] part with the result. So at least L ^= 4 would make sense as L = L ^ 4. Actually current semantics of += for lists: L += var and L = L + var are different, so it seems to me they were not meant to be bound together.
If the intention is to insert an element, indeed it is confusing, for me at least: L[0:0] = "bb" -> ["b","b","aa"]
On Tue, Jun 12, 2018 at 11:08 AM Mikhail V <mikhailwas@gmail.com> wrote:
The "/" operator means divide. How do you divide a list? The "^" operator means exclusive-or. Again, strange concept for a list. Yes, there are examples of Python (mis)using operators like "%" for string interpolation. Note the difficulties that caused; we now have 3 ways to interpolate. It looked cool at the time, but happily we have f-strings now.
One is mutation, the other isn't, but aside from that, the result is equivalent.
Thats, like, just your opinion, Man. Kidding aside, I think you'll find it's natural once you get used to slice assignment. The slice assignment iterates over the right-hand argument and inserts each element. It overwrites the existing slice if any. In this example, you've mixed issues. It looks like you've gotten confused by string iteration. When you loop a string, you get 1-len strings.
On Wed, Jun 13, 2018 at 5:25 AM, Michael Selik <mike@selik.org> wrote:
I agree about XORing a list, but dividing a list could conceivably be implemented to split a list into parts. For instance: low, high = list(range(10)) / 2 But it wouldn't mean "insert". Also, I can't imagine an augmented division operator being useful, but others may disagree. ChrisA
On Tue, Jun 12, 2018 at 10:25 PM, Michael Selik <mike@selik.org> wrote:
Allowed standard characters are so - they already mean something and there are so few (even less if throw away ugly looking). For me the plus character + means sum of numbers. So for an array: A += 1 Means for me unequivocally increment each element of array by 1. And this: A += B Means for me increment each element of A by corresponding values of B. BTW that is how plus operator works for Numpy arrays. So I don't think it is a precise thing how logical this or that character suits an operation. Everybody understands that overloading of operators is for convenience and it's just a shortcut for some frequent usage. As I understand, overloading some operators for certain object type (list in this case) has relatively low cost in terms of implementation. As for me - I'm fine with append() method and use it all the time. Frankly speaking, I was working on another syntax idea and it turned out that actually augmented operators may be very useful for that particular syntax design.
You're right of course. I was just confused by the fact that L += "aa" # works L = L + "aa" # gives TypeError But for consistent types both should work the same.
I think you'll find it's natural once you get used to slice assignment.
I use slice assignment all the time with Numpy arrays, though for list element appending I prefer append() method. M
On 6/12/2018 10:54 AM, Mikhail V wrote:
...
Note that there is a trick to 'insert' an element with slicing syntax, e.g.:
This is not a 'trick'. It is a particular case of a general operation: replacing a length m slice of a list with a sequence of length n. Both m and n can be 0. The replacement sequence can be any iterable.
In these examples, m and n are 0 and 1.
The trick is to put brackets around the element and so it works as insert().
Again, not a trick. Putting brackets around the element makes it sequence of length 1. To possible be less confusing, you could use (,)
Though additional brackets look really confusing for this purpose, so I don't feel like using this seriously.
Learning about lists means learning about slice assignment: replace a sublist with another sequence. -- Terry Jan Reedy
On Tue, Jun 12, 2018 at 10:26 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Yes I see, actually that is what I am saying - slice assignment has _replace_ semantics. If that would be a list method, I suppose it would work something like: L = L.replace(begin, end, L2) So that makes perfect sense. But returning to append/insert. So appending of course is extremely frequent operation, inserting in the beginning of the list may be also not rare. Concatenation in my opinion might be not so frequent. Writing values from one array (or slice) to another is quite frequent operation, that means slices of the same size. As for _replacing_ slices, of different size, I personally never used it. OTOH it smells like generalization. Anyway, I prefer to look at it through 'syntax glass'. Here is something ubiquitous: L = L.append("string1") L = L.append("string2") L = L.append("string3") L = L.append([1,2,3]) Could be written: L ^= "string1" L ^= "string2" L ^= "string3" L ^= [1,2,3] These kind of things are basically in every second python script. I'm ok with append() actually, but it just asks for a shortcut. For the ^ character - yes it looks strange. And writing it like this: L += ["string1"] L += ["string2"] L += ["string3"] L += [[1,2,3]] Sorry, no, that would be too much for me, also this needs some real mental training so as not to misinterpret these. Append() is just fine, though it's a pity there is no shortcut operator. M
On Tue, Jun 12, 2018 at 4:43 PM Mikhail V <mikhailwas@gmail.com> wrote:
inserting in the beginning of the list may be also not rare.
Inserting in the beginning of a list is *slow* and should be rare. If you want to append to the left-side of a list, you should use a deque. Check out ``collections.deque`` for your insert(0, x) needs.
^ is also used in regexes for matching the *beginning* of a string... Realistically, I don't think this proposal would be added, but if it were, ^ would be a horrible choice. That being said, I do understand the feeling of half your code being calls to .append or .extend. You could always do: L += 'string1' Only thing extra is the comma at the end; it converts the single value to a tuple, which is then added to the list. As for the use of +, I think that's mostly opinion. Many other languages use + for concatenation, so Python's hardly on its own here. On June 12, 2018 6:43:18 PM Mikhail V <mikhailwas@gmail.com> wrote:
Mikhail V wrote:
I don't think it's all that confusing. It looks a bit cluttered when the thing being inserted is itself a list literal, but that seems like a rare case of something that's not all that coommon in the first place. My feeling is that inserting is not a frequent enough operation to warrant having its own operator, especially not when there is already a syntax that does the same thing.
Is there some technical problem with implementing this?
Yes, it doesn't fit into the current scheme for augmented assignment operators. There are no special methods for combining augmented assignments with slicing -- you couldn't implement this just by adding an __ixor__ method to the list type. There would need to be a new special method for "in-place xor with slice", and the compiler would have to recognise this combination and emit special bytecode for it. That would raise the question of why ^= is getting this special treatment but not any of the other augmented assignments, and why not "in-place operation with attribute" as well, and potentially we would end up with two new entire sets of special methods for different flavours of augmented assignments. I really don't think we want to go there. -- Greg
On Wed, Jun 13, 2018 at 2:15 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Mikhail V wrote:
Depends on what you count as 'insert' - append is one case of insert ;) (logically seen) Sorry for repeating myself, the idea was that the default meaning is append(), i.e. normal operator usage on list: L1 = L2 ^ item - would be same as L1 = L2.append(item) But hope you get the point - if an operator for append was added, then it would be a bit sad that it cannot be used for inserting by slicing, namely these two forms: L ^= item #append(item) L[i:j] ^= item #insert(i, item) instead of i:j items would be IMO nice to have and it 'd cover both insert and append. But if you say that special-casing of [i:j] here would be hard to implement, then maybe insert() idea should be dropped.
As said, I don't insist on ^ operator. I would find ">>" or "<<" or "|" ok as well: L <<= "foo" L1 = L2 << "foo" L |= "foo" L1 = L2 | "foo" But it would raise same questions. As for relation 'by sense', that may be too opinion based. Vertical bar "|" may be somewhat related - IIRC in some contexts it is used as element separator. Just hope the judgement is not like "the symbol looks strange - therefore the feature is not needed". M
On Thu, Jun 14, 2018 at 12:04 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Not sure exactly what your intention here is, because list.append mutates the list and returns None. Does "L2 ^ item" mutate L2 in place, or does it construct a new list? If it mutates in place, does it return the same list? Or if doesn't, how is it different from "L2 + [item]", which is a much more logical spelling of list addition? ChrisA
On Thu, Jun 14, 2018 at 12:40 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Okay. Now it all is coherent and makes perfect sense... but you're offering alternative spellings for what we can already do. The only improvement compared to the + operator is that you don't need to surround the operand in brackets; in return, it's less general, being unable to add multiple elements to the list. The only improvement compared to .append is that it's an operator. There's no connection to exclusive-or, and there's not a lot of "but it's intuitive" here (cf Path division). ChrisA
On Wed, Jun 13, 2018 at 5:46 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Jun 14, 2018 at 12:40 AM, Mikhail V <mikhailwas@gmail.com> wrote:
Exactly, IMO it's important to make distinction for the item append. Not only for practical reason, but for semantical distinction, just like append() vs extend() distinction. Otherwise if += is used as append(), it creates an illusion that += _is_ append().
Mikhail V wrote:
Okay, that achieves an in-place append, but it's not exactly obvious to the unenlightened what it does, whereas append() is pretty self-explanatory. Also, using the slice version to do an insert L[i:i] ^= item is not as efficient as it looks like it should be, because it creates an empty list, appends the item to it and then splices that back into the list. And you have to write the index twice. Whereas L.insert(i, item) doesn't have any of those problems, and again is mostly self-explanatory. Python is not Perl. Not every operation that you use more than once in a blue moon needs to have its own operator. -- Greg
On Thu, Jun 14, 2018 at 2:58 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Sure, I use append() only and obviously would recommend to do so. I am not sure though everything's so simple here. Yet by writing examples here I've made typos several times, like L = L.append() - so strong is the influence of functional programming. I don't like cryptic operators, although such cases make me think in-place operations need some specialty in their syntax and augmented assignment could provide something positive in this regard. Another point is that people do like augmented operators much and for the append - there are so many advises like: hey, use L += [item] ! But Imo this just makes things worse from both practical and semantics POV (still I find it hard to explain why exactly, that's just some gut feeling). Not to argue, since I personally wouldn't benefit much from the idea, but it just seems important, more than just 'cosmetics'.
Efficiency is important ... where it is important.
On Wed, Jun 13, 2018 at 6:45 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Another point is that people do like augmented operators much and for the append - there are so many advises like: hey, use L += [item] !
another data point -- in teaching, a number of newbie students do exactly that. Actually, they do: a_list += an_item and find it does not do what they want, and then the get confused, and I show that they need: a_list += [an_item] or a_list.append(an_item) (this gets particularly confusing them an_item is, itself, a sequence (a string is really common). So it would be nice to have an operator version of append -- but given the limited number of operators, and their usual uses, I suspect it would cause even more confusion... But throwing it out there, how about (ab)using the mat_mul operator: a_list @= an_item Another note: One of the major motivations for augmented assignment was being able to support in-place operations for numpy: an_array += something which was MUCH nicer notation that was was required: np.sum(an_array, something, out=an_array) That is a much bigger win than going from .append() to an operator. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Jun 14, 2018 at 7:40 PM, Chris Barker <chris.barker@noaa.gov> wrote:
On Wed, Jun 13, 2018 at 6:45 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Yes, none of the oprators fits well by sense. But which symbol would fit here if you could choose *any* symbol? I've picked the caret ^ for two reasons: 1. It is originally associated with the insertion mark.
So there is some relation with "insert" operation. But of course this makes sense mainly for people who have some editorial or typography background, (such as myself). So for me it makes perfect sense. 2. The symbol ^ itself looks quite 'gentle' and to my eye, is not too distracting (at least less than some other operators). So e.g. I find that vertical bar | causes some bad eye straining effect.
Agreed.
There's nothing wrong with your ideas if you were designing a language from scratch. However, Python has a long history and many tools and uses for the same operators you are considering. And even has a current "insert" operator (slice assignment). When adding a new feature, you need to consider if you're duplicating current functionality and if a 2nd "obvious" way is worth breaking the Zen. Also, consider whether your "intuitive" design is following or breaking a de-facto standard established elsewhere in the language. For example, if you're making a file format decoder, you should create functions "load" and "dump" to behave like every other file format decoder module. Don't re-invent or we'll end up with too many "standards" and the language will become hard to remember. In case I need to clarify: 1. You're duplicating current clear and more flexible syntax. 2. Your proposed operators are confusing when compared with their meanings elsewhere.
On Fri, Jun 15, 2018 at 1:04 AM, Michael Selik <mike@selik.org> wrote: [..]
what haven't we repeated in this thread yet? Motivation was explained. About confusion probably haven't discussed : so Xor ^ works on sets, IIRC for finding union without common elements. That's one point for potential confusion - probably expect it to work with list. Sounds probable? Other operator, e.g. bitshift <<, I dont' think it has potential for confusion. @ operator - currently 'free'. But maybe there is some reserved plans for lists and these operators as well - I don't know. IIRC some time ago Steven D'Aprano proposed something with sets and some of the operators (sorry in advance if that is wrong info, but I\m almost sure there was a related mathematical discussion with sets involved).
On Thu, Jun 14, 2018, 7:24 PM Mikhail V <mikhailwas@gmail.com> wrote:
what haven't we repeated in this thread yet? Motivation was explained.
You have repeated your explanations a few times. It isn't convincing. It seems to me that your main complaint is that strings are iterable, though you haven't expressed it as such. During slice assignment, you're surprised that the string is a sequence of characters. I understand, but I don't find that confusion a compelling reason to add a new operator. About confusion probably haven't discussed : so Xor ^ works on sets,
IIRC for finding union without common elements.
I wrote about this in an earlier email, which you didn't reply to. You're correct that xor doesn't make sense with lists, especially not between a list and a single element. Again, though other operators have been given different meanings in special contexts, those decisions should be seen as abnormal, not an excuse to keep creating more special cases.
Sorry, I forgot that you dropped the suggestion to make it an insert operator and are only asking for an append operator. I see no benefit to this, because += already is an elegant way to extend a list, which is more flexible than append. Yes, if the right-hand is an iterable and should be appended as a single element, you'll need to enclose it in a single-element container. This is true for strings, lists, sets, whatever. It's natural and is not a "trick". If you would like to prove the need for this operator, one piece of evidence you can provide is a count of the number of times someone writes "list.append" for an iterable vs "+=" and encloses a str or other type in a throw-away list to effectively append. If the latter habit is common, that's evidence that the language may need to be improved. Why don't you search GitHub projects to collect some statistics? On Thu, Jun 14, 2018, 7:40 PM Michael Selik <mike@selik.org> wrote:
On Fri, Jun 15, 2018 at 5:51 AM, Michael Selik <mike@selik.org> wrote:
That's strange idea - there is no doubt that one would use list.append() and most probably it is the case statistically. So the question would be "what is wrong with list.append()?" And as said many times, there is nothing wrong, but a lot of people seem to want an in-place operator for this purpose. And I can understand this, because: 1. append() is _ubiquitous_ 2. in-place assignment form makes some emphasis on mutating, in contrast to method call. That's it. So instead of a method call one gets a clean element on the right-hand and (hopefully) emphasis on the in-place nature of operation. A quick google search shows some tendency: https://stackoverflow.com/a/2022044/4157407 https://stackoverflow.com/a/28119966/4157407 So you shoudn't explain it to _me_ - I don't see other significant convincing points, and unless this gets support here - I am not interested in continuing.
encouraging to mimic append() via += operator is bad practice. In the above links to SO, people try to tell the same, but that is not something that can be easily explained on paper. Now add here the constant wish for using in-place operator - and there will be on-going confusion. Regardless of how natural or general it is, it's something that should be avoided. Only in this discussion thread I had 3 different advices to use: L += ["aa"] L += ("aa",) L += "aa", But you should not give it to _me_ - i use the correct form only: L.append("aa")
On Sat, Jun 16, 2018 at 1:48 AM, Mikhail V <mikhailwas@gmail.com> wrote:
How so? You can write "x += 1" with integers, and that doesn't mutate; but if you write "x.some_method()", doing nothing with the return value, it's fairly obvious that it's going to have side effects, most likely to mutate the object. Augmented assignment is no better than a method at that. ChrisA
On Fri, Jun 15, 2018 at 6:54 PM, Chris Angelico <rosuav@gmail.com> wrote:
How it would be obvious unless you test it or already have learned by heart that x.some_method() is in-place? For a list variable you might expect it probably, and if you already aware of mutability, etc. It s just very uncommon to see standalone statements like: x.method() for me it came into habit to think that it lacks the left-hand part and =. Of course augmented assignment is not a panacea because it is limited only to one operation, and the appeal of the operator itself is under question. As for x+=1 it is implementation detail - historical idea of such operators was mutating, so at least visually its not like a returning expression. and I am not sure about x.method() form - was it meant to hint to the user about anything? It seemed to me so when I started to learn Python, but its not.
On Fri, Jun 15, 2018 at 10:25 AM Mikhail V <mikhailwas@gmail.com> wrote:
very uncommon to see standalone statements like: x.method()
Python has many such mutation methods. It sounds like you're judging the frequency of code patterns across all languages instead of just Python. Even then, I don't think that's true. All OO languages that come to mind have that pattern frequently. As for x+=1 it is implementation detail - historical idea of such operators
was mutating, so at least visually its not like a returning expression.
Incorrect. The += operator was meant as an alias for ``x = x + 1``. The fact that it mutates a list is somewhat of a surprise. Some other languages make no distinction between mutation and reassignment. Perhaps you're thinking of one of those other languages.
Yes, it returns None to emphasize that it's a mutation. This is different from the so-called "fluent" design pattern where all mutation methods also return the original object, causing confusion about whether the return value is a copy or not.
Michael Selik wrote:
The += operator was meant as an alias for ``x = x + 1``. The fact that it mutates a list is somewhat of a surprise.
That's very much a matter of opinion. For every person who thinks this is a surprise, you can find another that thinks it's obvious that += should mutate a list, and is surprised by the fact that it works on immutable types at all.
Most languages, actually. I'm not aware of any other language with a += operator that has this dual interpretation of its meaning -- which is probably why almost everyone gets surprised by at least one of its meanings. :-) Maybe we should call it the "Spanish Inquisition operator". -- Greg
On Sat, Jun 16, 2018 at 01:06:45PM +1200, Greg Ewing wrote:
Given the ubiquity of += in C, where it works on numbers but not lists, and the general difficulty many people have in dealing with the difference between assignment and mutation, I think the ratio would be closer to 20:1 than 1:1. But regardless, I'm pretty sure that nobody expects this: py> t = ([], None) py> t[0] += [1] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment py> print(t) ([1], None)
surprised by at least one of its meanings. :-)
Maybe we should call it the "Spanish Inquisition operator".
:-) -- Steve
On Sat, Jun 16, 2018 at 9:09 PM, Steven D'Aprano <steve@pearwood.info> wrote:
The nearest you'd get to "adding to a list/array" in C would be adding to a pointer. But I don't see people writing code like this in Python:
:)
Agreed, although I can't remember this ever coming up outside of interactive work. When you do something at the interactive prompt, you can kinda feel that it should "rollback" on exception, and yet some part of it has happened. But in most scripts, that TypeError is going to pull you all the way out of the scope where 't' exists - frequently, it'll just straight-up terminate the script - so you won't often see this. ChrisA
On 6/15/18 1:25 PM, Mikhail V wrote:
For me if I see foo.bar() my assumption is that likely bar() is going to mutate foo, especially if it doesn't produce a return value, and this is the natural way to define something that mutates something. If it returns something, it might be just an accessor, or it might be a bit of both, easily a mutation that returns a status about the success/failure of the mutation. foo = bar would seem to be never a 'mutation', but always a rebinding of foo, even if 'bar' is an expression using foo. foo += bar, conceptually reads as foo = foo + bar, so the first impression is that it isn't going to mutate, but rebind, but it is possible that it does an inplace rebind so to me THAT is the case where I need to dig into the rules to see what it really does. This means that at quick glance given: list1 = list list1 += value it is unclear if list would have been changed by the +=, while a statement like list1.append(value) is more clearly an in place mutation so other names bound to the same object are expected to see the changes. -- Richard Damon
Now I have slightly different idea. How is about special-casing of this as a shortcut for append: L[] = item Namely just use the fact that empty slice is SyntaxError now. I understand this is totally different approach than operator overloading and maybe hard to implement, but I feel like it looks really appealing. And it is quite intuitive imo. For me the syntax reads like: "add new empty element and this element will be "item". No?
On Sat, Jun 16, 2018 at 3:02 AM, Chris Angelico <rosuav@gmail.com> wrote:
Is it like that in PHP?
I still haven't seen any compelling argument against the append method. -1 on introducing a new way to spell append.
By me - there is just nothing against append() method. But I see from various posts in the web and SO - people just want to spell it compact, and keep the 'item' part clean of brackets. And that kind of makes sense.
On Fri, Jun 15, 2018, 5:24 PM Mikhail V <mikhailwas@gmail.com> wrote:
there is just nothing against append() method.
Then why break the Zen: there should be only one obvious way? But I see from various posts in the web and SO - people just want to spell
it compact, and keep the 'item' part clean of brackets.
If you're going to cite things, please link to them.
On Sat, Jun 16, 2018 at 3:26 AM, Michael Selik <mike@selik.org> wrote:
I think the question could be applied to 99% proposals
Want to force me to work? :) I made pair of links already in previous posts, not directly telling concrete wishes, but shows approximately the picture. Something more concrete maybe: https://stackoverflow.com/a/3653314/4157407 https://stackoverflow.com/q/13818992/4157407
On Sat, Jun 16, 2018 at 3:47 AM, Michael Selik <mike@selik.org> wrote:
One of those links was discussing extend, not append.
Yes and so what? Does this makes it automatically not related to the wish to choose more compact spelling, despite it is not recommended way. What is different with append? Similar posts are in topics about append and anything else actually, e.g.: i = i +1 i +=1 You can find enough evidence that people prefer i += 1.
The other wanted a repeated append, best solved by a list comprehension.
I think yuo should reread that one more thoroughly - I'll even paste here the text: """ When appending longer statements to a list, I feel append becomes awkward to read [...] Though append works just fine, I feel it is less clear than: mylist[i] = 2*math.pi*radius*math.cos(phi[i]) """" So your claim this ^ is not relevant?! Seriously I am starting to get tired of that style of conversation. I provided you links - you are not pleased again.
On Fri, Jun 15, 2018, 6:18 PM Mikhail V <mikhailwas@gmail.com> wrote:
Luckily for extend, it's similar to the "obvious" semantics of ``a += b`` which is ``a = a + b``. Unfortunately for append, there's nothing quite like it among the operators.
mylist[i] = 2*math.pi*radius*math.cos(phi[i])
So your claim this ^ is not relevant?!
I read that sentence. No, it was not relevant, because the best answer was to teach that person about list comprehensions, not to offer a new syntax. Seriously I am starting to get tired of that style of conversation.
To be honest, I'm getting a little tired myself. I am trying to politely suggest ways to strengthen your proposal even though I disagree with it. I provided you links - you are not pleased again.
Are you aware that modifying the language is difficult, time consuming, and the folks that do it aren't paid for their work? Further, any change is likely to increase the maintenance burden on these same volunteers. Even worse, tens of thousands of teachers will need to add more time to their lesson plans to explain new features. Book authors will need to issue errata and revised versions. If you add that all together, in a sense, changing the parser to expand the syntax would cost millions of dollars. Is that worth the change?
On 16Jun2018 02:42, Mikhail V <mikhailwas@gmail.com> wrote:
Now we're just making typing errors into working code. Also, that isn't an empty slice. That's a _missing_ slice. An empty slice has zero length. While genuinely new syntax needs to land in such a gap (because otherwise it will break working code), new syntax needs a much highly value than new meanings for existing operators. Some things _should_ be syntax errors. Particularly things which may be typing errors. Suppose I'd meant to type: L[0] = item Silent breakage, requiring runtime debugging.
The term "new empty element" is a nonsense term to me. If you mean "replace an empty slice at the end of the list with a new element", that can already be written: L[len(L):len(L)]=[9] Cumbersome, I accept. But I've got a .append method. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, Jun 16, 2018 at 4:44 AM, Cameron Simpson <cs@cskk.id.au> wrote:
On 16Jun2018 02:42, Mikhail V <mikhailwas@gmail.com> wrote:
Not sure that it's different from any other situation: e.g. if by mistake I will write: L[i+1] instead of L[i+2] Visual difference between L[] = L[1] = is big enough to reduce the typing error. But maybe I did not understand your example case. FWIW in general, claims about possible typing errors by introducing this or that syntax are speculative. Main source of typos usually is high similarity of spellings and characters, or initial obscurity of spelling - e.g. excessive punctuation. Does it apply here?
I just say that I find this syntax is more intuitive than the idea with operator. Not sure that we need to start semantics nitpicking. For example, such code: L = [] L[] = x L[] = y imo has more chance to be understood correctly than e.g.: L = [] L ^= x L ^= y By L[] there is some mnemonical hint because [] is used to create new empty list. Plus it does not introduce overloading of the operator. And overloading has weakness in this - e.g. " var1 += var2 " does not have mnemonics, other than + character (it could be two integers as well). So if L[] is found somwhere far from initialisation - it may be a good aid. It makes it more clear what is happening, compared to augmented operator.
On Sat, Jun 16, 2018, 10:22 AM Mikhail V <mikhailwas@gmail.com> wrote:
Plus it does not introduce overloading of the operator.
Now you're critizing duck typing. And overloading has weakness in this - e.g. " var1 += var2 " does not
have mnemonics, other than + character (it could be two integers as well).
That's an advantage, not disadvantage.
On 16Jun2018 20:21, Mikhail V <mikhailwas@gmail.com> wrote:
Well, many typing errors are invalid syntax. When that happens you find out immediately.
This is true: many other typing errors are not syntax errors.
No, you understand the example; we differ in our estimation of likelihoods and costs to mistakes. Particularly, I dislike "silent breakage", which aan be much harder to fix because it shows as incorrect behaviour far from the error (eg counters being slightly wrong leading to misbehaviour in things depending on the counter). Assuming we are lucky enough for the misbehaviour to be obvious. But there is also the point that _every_ new piece of syntax reduces the surface of "invalid syntax that can catch simple mistakes". So new syntax tends to requires a higher perceived benefit than, say, a new feature on a class/type. Such as your suggestions about having lists support more operators i.e. "^" to mediate list insertion.
FWIW in general, claims about possible typing errors by introducing this or that syntax are speculative.
Certainly.
Unsure, depends on the programer.
Well, as someone else pointed out, PHP has this notation for append. It is quite convenient. (In fact, it is one of the very few convenient things in PHP.) However, PHP is generally not a winning source of ideas in the Python world. Certainly I was _very_ glad to not be suffering with PHP once I left my previous job. There were many many things to like about my previous job, but PHP was not one of them. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, Jun 16, 2018 at 08:21:42PM +0300, Mikhail V wrote:
Should be written as L = [x, y].
I disagree. The first syntax L[] = x looks so similar to L[:] assignment that I keep reading it as "set the list L to a single item x". It certainly doesn't look like an append operation. The second at least looks like a mutation on L.
By L[] there is some mnemonical hint because [] is used to create new empty list.
How is that a hint? What is the connection between "append an item" and "create a new empty list"?
So if L[] is found somwhere far from initialisation - it may be a good aid. It makes it more clear what is happening, compared to augmented operator.
I don't think so. Here is a radical thought... why don't we give lists a method that inserts items at the end of the list? We could call it something like "append", and then instead of hoping people guess what the syntax does, they can just look up the name of the method? L.append(x) might work. *wink* -- Steve
On Sun, Jun 17, 2018 at 2:52 AM, Steven D'Aprano <steve@pearwood.info> wrote:
On Sat, Jun 16, 2018 at 08:21:42PM +0300, Mikhail V wrote:
Where did I say it has _direct_ connection? It has some associative connection - 'new item', 'special index case'. L = [] is new list which is supposed to be filled with something. And it has references to existing syntax, e.g. slice assignment, or adding dictionary item can be written as : mydict[key] = value so: mylist[] = item is not THAT far. I would even say it's very close - but I'm pretty sure you can find something against this as well.
Exercising in wit? How about: let's assume most people can't understand even the simplest new feature. So instead of giving clean compact assignment syntax for frequent operation, let's force less readable method call everywhere, which also might be harder to remember than one index case []. *wink* Maybe force .extend() for lists and .update() for dicts everywhere? To preserve poor new users from 'unintuitive' assignment syntax. But how you would do that?
On Sun, Jun 17, 2018 at 02:43:16PM +0300, Mikhail V wrote:
I didn't mention "direct" connection. You said there is a mnemonic from "the empty list" to "L[] is used for insert". I don't know what that mnemonic is, and your description below doesn't make sense to me. Insert/append doesn't just work on empty lists, so the connection between "empty list" and inserting/appending is pretty tenuous. Aside from the presence of a list, I don't see any connection at all. If we think about this example: L = ["starting", "values", "go", "here"] for item in extra_values: L[] = item do you still see a connection between a non-empty list and L[] used for append?
It has some associative connection - 'new item', 'special index case'. L = [] is new list which is supposed to be filled with something.
I don't see any connection between "new item" and "special index case" either.
I would expect mylist[] = value to mean that the contents of mylist is replaced with a single item. And then I would wonder why this is so special that it needs dedicated syntax, when we can already do it with existing syntax: mylist[:] = [item] The idea that assignment to mylist[] means "append to the end of the list" would never cross my mind in a million years.
Compact it might be: L[] = x # seven characters, including spaces L.append(x) # eleven characters but I think that calling it "clean" is inappropriate. Slice notation is already one of the trickier things for beginners to learn, and this adds extra complexity for not much benefit. As for "frequent operation", there are lots of frequent operations in Python. Does every one of them deserve special syntax to make it clean? I just opened one of my modules at random, and I don't have a single append in that module, but I have 14 calls to string.startswith and nine calls to kwargs.pop. Appending to a list might be common, but I don't see that it is either common enough or important enough to add additional syntax.
let's force less readable method call everywhere,
I think that describing a short, self-descriptive method call like append as "less readable" demonstrates a deep and fundamental gulf between the style you prefer and what the rest of us prefer. Mikhail, sometimes I wonder if you would be happier using Perl rather than Python. Like the Perl community, you seem to have a desire for syntax which most of us see as terse and cryptic over self-descriptive method names. But even Perl doesn't (so far as I know) give us special syntax to append to an array. As far as I know, the standard way to append to an array @arr in Perl is: push(@arr, item); -- Steve
Mikhail, this thread is getting quite long, and difficult to follow. It's quite clear that a new operator won't be introduced for list insertion. Furthermore, this thread has now become a hard-to-follow and impossible-to-participate-in debate. If you have other ideas, could you maybe formulate them unambiguously and post them in a new thread? With examples. I'm interested in reading about new ideas, but not rhetorics and argumentations. Best wishes, Pål Grønås Drange
On Sun, Jun 17, 2018 at 11:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
kwargs.pop("some_key") could plausibly be spelled del kwargs["some_key"] if del were (like yield) upgraded to expression. Whether that is an improvement or not, I don't know, but at least it's logical. ChrisA
Mikhail V wrote:
You must be looking at a very limited and non-typical corpus of Python code. Mutating method calls are extremely common in most Python code I've seen. You seem to be trying to reason about Python as though it were intended to be used in a functional style, but it's not. Making guesses about it based on how something would be done in a functinal language will get you nowhere.
I am not sure about x.method() form - was it meant to hint to the user about anything? It seemed to me so when I started to learn Python, but its not.
The fact that something is a method does not, and was never intended to, imply anytbing about whether it is mutating. However, there *is* a fairly easy way to tell, most of the time, when you're *reading* code. There's a convention that mutating methods don't return anything other than None, so mutating method calls reveal themselves by the fact that the result is not used. Conversely, a method call whose result *is* used is most likely non-mutating. (It's possible that it could both return a value and have a side effect, but that's frowned upon, and you'll pretty much never see it in any builtin or stdlib API.) As for writing code, you just have to rely on memory and documentation, just like you do with most aspects of any language and its libraries.
Mikhail V wrote:
But if you say that special-casing of [i:j] here would be hard to implement, then maybe insert() idea should be dropped.
Since I wrote that I realised that it's not true -- given an infix ^ operator like you propose, the in-place version of it would actually give the desired result. However, I still don't think that either insert or append are frequent enough operations to warrant having their own operators. It's true that append is a lot more common than insert, but usually it's a mutating append that you want, not creating a new list with one more item on the end, which is what your ^ operator does. -- Greg
participants (14)
-
Cameron Simpson -
Chris Angelico -
Chris Barker -
Clint Hepner -
Greg Ewing -
Jelle Zijlstra -
Michael Selik -
Mikhail V -
Pål Grønås Drange -
Richard Damon -
Ryan Gonzalez -
Serhiy Storchaka -
Steven D'Aprano -
Terry Reedy