Allow popping of slices
The `pop` method of built-in sequences is basically an atomic version of val = self[pos] del self[pos] return val If this behavior was extended to the case where `pos` is a slice, you could write things like: def cut_deck(deck, pos): deck.extend(deck.pop(slice(0, pos))) def bfs(roots): depth, frontier = 0, list(roots) while frontier: depth += 1 for item in frontier.pop(slice(None)): ... frontier.append(...) ... Similar functionality is found in many other languages (e.g. Perl and JavaScript's `splice`). I think it's useful not just because it's more concise, but because it's linear/reversible: it moves data rather than duplicating and then destroying it, which makes it less prone to bugs. The syntax is a bit odd since you have to construct the slice by hand. Here are three solutions for that from least to most extravagant: 1. Don't worry about it. It's still useful, and the syntax, though verbose, makes sense. (The "reference implementation" of pop is literally unchanged.) 2. Give pop methods a __getitem__ that does the same thing as __call__, so you can write xs.pop[-1] or xs.pop[:]. 3. Promote del statements to expressions that return the same values as the underlying __delitem__, __delattr__, etc., and make those methods of built-in types return the thing that was deleted. (Or introduce __popitem__, __popattr__, etc. which return a value.) -- Ben
On Mon, Jun 04, 2018 at 03:23:07PM -0700, Ben Rudiak-Gould wrote:
Aside from the atomicness, for testing we can subclass list: # Untested class MyList(list): def pop(self, pos): if isinstance(pos, slice): temp = self[pos] del self[pos] return temp return super().pop(pos) Is that what you have in mind?
I'm not sure that's an advantage over: deck[:] = deck[pos:] + deck[:pos] but I suppose one might be faster or slower than the other. But I think the version with slices is much more clear.
If that's a breadth-first search, I've never seen it written like that before. The classic bfs algorithm is at Wikipedia (conveniently written in Python): https://en.wikipedia.org/wiki/Breadth-first_search#Pseudocode and yours is very different. I'm not saying yours is wrong, but its not obviously right either. It might help your argument if you show equivalent (but working) code that doesn't rely on popping a slice.
Similar functionality is found in many other languages (e.g. Perl and JavaScript's `splice`).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj... Do you have a link for the Perl version?
I don't see how that is possible. You still have to move the data out of the sequence into a new sequence before deleting it from the original. Maybe there are optimizations if the slice is at the end of the sequence, but in general you have to make a copy of the items. (Well, not the items themselves, just the references to them.)
Indeed. If this functionality can be justified, we could start with this, and think about a neater syntax later (if required).
2. Give pop methods a __getitem__ that does the same thing as __call__, so you can write xs.pop[-1] or xs.pop[:].
That's interesting. But it would mean that pop, and only pop, would allow pop(n) and pop[n] to be the same thing. That's going to confuse people who wonder why they can't call other methods like that: mylist.pop[0] # okay mylist.append[item] # fails and why they can't use slice syntax in the round-bracket call syntax: mylist.pop[1:-1] # okay mylist.pop(1:-1) # not okay While I'm intrigued by this, I think it will be too confusing.
I don't get how this allows us to pass slices to pop. You missed one, allow slice literals: https://mail.python.org/pipermail/python-ideas/2015-June/034086.html -- Steve
On Mon, Jun 4, 2018 at 5:11 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Yes. (I used almost exactly that to test my examples.)
That's fair. I didn't spend as long creating examples as I probably should've.
It might help your argument if you show equivalent (but working) code that doesn't rely on popping a slice.
When I have a collection of items and I want to consume them, process them, and produce a new collection of items, I often find myself writing something along the lines of items2 = [] for item in items: ... items2.append(...) ... items = items2 del items2 The last two statements aren't strictly necessary, but omitting them is a bug factory in my experience; it's too easy to use the wrong variable in subsequent code. When the loop is simple enough I can write items = [... for item in items] and when it's complicated enough it probably makes sense to split it into a separate function. But I've many times wished that I could write for item in items.pop_all(): ... items.append(...) ... This proposal uses pop(slice(None)) instead, because it's a natural extension of the existing meaning of that method. My bfs function was a breadth-first search. The outer loop runs once for each depth, and the inner loop once for each item at the current depth (the frontier). The code in the Wikipedia article uses a FIFO queue instead and has just one loop (but doesn't explicitly track the depth).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...
Do you have a link for the Perl version?
https://perldoc.perl.org/functions/splice.html It's slightly more versatile: it essentially does old = ARRAY[OFFSET:OFFSET+LENGTH] ARRAY[OFFSET:OFFSET+LENGTH] = LIST return old The special case LIST = [] is what I'm suggesting for the pop method.
Under the hood it copies data because that's how von Neumann machines work, but from the Python programmer's perspective it moves it. (Similarly, std::swap(x, y) in C++ probably does { t = x; x = y; y = t; } internally, but it looks like a swap from the programmer's perspective.)
It doesn't; it basically makes del the new pop. It's almost pop already, except that it doesn't return the deleted value. I wasn't seriously proposing this, although I do like the idea. I don't think it reaches the usefulness threshold for new syntax. Also, del foo[:] would be slower if it had to move the deleted items into a new list whether or not the caller had any interest in them. That's why I suggested that del expressions call __popXXX__ instead, while del statements continue to call __delXXX__; but now it's getting complicated. Oh well. -- Ben
On Tuesday, June 5, 2018 at 12:19:53 AM UTC-7, Ben Rudiak-Gould wrote:
Isn't it equally easy to write: def foo(x): ... items = [foo(x) for x in items] I don't understand the desire to write this as a loop over a ``items.pop(slice(None))``. You can simply use the same loop body, replacing the ``for x in items:`` with ``def blah(x):`` and ``items.append(y)`` with ``return y``. Low typing effort.
On Mon, Jun 04, 2018 at 03:23:07PM -0700, Ben Rudiak-Gould wrote:
Aside from the atomicness, for testing we can subclass list: # Untested class MyList(list): def pop(self, pos): if isinstance(pos, slice): temp = self[pos] del self[pos] return temp return super().pop(pos) Is that what you have in mind?
I'm not sure that's an advantage over: deck[:] = deck[pos:] + deck[:pos] but I suppose one might be faster or slower than the other. But I think the version with slices is much more clear.
If that's a breadth-first search, I've never seen it written like that before. The classic bfs algorithm is at Wikipedia (conveniently written in Python): https://en.wikipedia.org/wiki/Breadth-first_search#Pseudocode and yours is very different. I'm not saying yours is wrong, but its not obviously right either. It might help your argument if you show equivalent (but working) code that doesn't rely on popping a slice.
Similar functionality is found in many other languages (e.g. Perl and JavaScript's `splice`).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj... Do you have a link for the Perl version?
I don't see how that is possible. You still have to move the data out of the sequence into a new sequence before deleting it from the original. Maybe there are optimizations if the slice is at the end of the sequence, but in general you have to make a copy of the items. (Well, not the items themselves, just the references to them.)
Indeed. If this functionality can be justified, we could start with this, and think about a neater syntax later (if required).
2. Give pop methods a __getitem__ that does the same thing as __call__, so you can write xs.pop[-1] or xs.pop[:].
That's interesting. But it would mean that pop, and only pop, would allow pop(n) and pop[n] to be the same thing. That's going to confuse people who wonder why they can't call other methods like that: mylist.pop[0] # okay mylist.append[item] # fails and why they can't use slice syntax in the round-bracket call syntax: mylist.pop[1:-1] # okay mylist.pop(1:-1) # not okay While I'm intrigued by this, I think it will be too confusing.
I don't get how this allows us to pass slices to pop. You missed one, allow slice literals: https://mail.python.org/pipermail/python-ideas/2015-June/034086.html -- Steve
On Mon, Jun 4, 2018 at 5:11 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Yes. (I used almost exactly that to test my examples.)
That's fair. I didn't spend as long creating examples as I probably should've.
It might help your argument if you show equivalent (but working) code that doesn't rely on popping a slice.
When I have a collection of items and I want to consume them, process them, and produce a new collection of items, I often find myself writing something along the lines of items2 = [] for item in items: ... items2.append(...) ... items = items2 del items2 The last two statements aren't strictly necessary, but omitting them is a bug factory in my experience; it's too easy to use the wrong variable in subsequent code. When the loop is simple enough I can write items = [... for item in items] and when it's complicated enough it probably makes sense to split it into a separate function. But I've many times wished that I could write for item in items.pop_all(): ... items.append(...) ... This proposal uses pop(slice(None)) instead, because it's a natural extension of the existing meaning of that method. My bfs function was a breadth-first search. The outer loop runs once for each depth, and the inner loop once for each item at the current depth (the frontier). The code in the Wikipedia article uses a FIFO queue instead and has just one loop (but doesn't explicitly track the depth).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...
Do you have a link for the Perl version?
https://perldoc.perl.org/functions/splice.html It's slightly more versatile: it essentially does old = ARRAY[OFFSET:OFFSET+LENGTH] ARRAY[OFFSET:OFFSET+LENGTH] = LIST return old The special case LIST = [] is what I'm suggesting for the pop method.
Under the hood it copies data because that's how von Neumann machines work, but from the Python programmer's perspective it moves it. (Similarly, std::swap(x, y) in C++ probably does { t = x; x = y; y = t; } internally, but it looks like a swap from the programmer's perspective.)
It doesn't; it basically makes del the new pop. It's almost pop already, except that it doesn't return the deleted value. I wasn't seriously proposing this, although I do like the idea. I don't think it reaches the usefulness threshold for new syntax. Also, del foo[:] would be slower if it had to move the deleted items into a new list whether or not the caller had any interest in them. That's why I suggested that del expressions call __popXXX__ instead, while del statements continue to call __delXXX__; but now it's getting complicated. Oh well. -- Ben
On Tuesday, June 5, 2018 at 12:19:53 AM UTC-7, Ben Rudiak-Gould wrote:
Isn't it equally easy to write: def foo(x): ... items = [foo(x) for x in items] I don't understand the desire to write this as a loop over a ``items.pop(slice(None))``. You can simply use the same loop body, replacing the ``for x in items:`` with ``def blah(x):`` and ``items.append(y)`` with ``return y``. Low typing effort.
participants (3)
-
Ben Rudiak-Gould -
Michael Selik -
Steven D'Aprano