Re: [Python-ideas] slice[] to get more complex slices
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to add additional syntax to the language for that. For example, one could say that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not a Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4] of slices too.
On 23 July 2018 at 11:31, Jeroen Demeyer J.Demeyer@ugent.be wrote:
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to add additional syntax to the language for that. For example, one could say that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not a Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4] of slices too.
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
Paul
That would be even more direct....but it require syntax support, usefull mainly for people doing multdim complex slicing (i.e numpy users). I may be wrong, but I do not see it gain much support outside numpy...
slice[....] is probably much more easy to swallow for standard python users, and almost as good imho. It reuse getitem, so it imply the produced slice will behaves exactly like it would if it was not stored/reused, and almost garantee it will be the case indeed (even if the slice syntax is extended)
Getting this to work including a new module would be nice. Eventually, having it standard is positive too, it means slice manipulation will become more standardised.
On Monday, July 23, 2018 at 12:32:04 PM UTC+2, Jeroen Demeyer wrote:
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to add additional syntax to the language for that. For example, one could say that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not a Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4] of slices too. _______________________________________________ Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Jul 23, 2018 at 7:24 AM, Paul Moore p.f.moore@gmail.com wrote:
On 23 July 2018 at 11:31, Jeroen Demeyer J.Demeyer@ugent.be wrote:
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to add additional syntax to the language for that. For example, one could say that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not a Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4] of slices too.
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
Paul
That involves initializing an instance, which doesn't serve any purpose in this case and I was hoping to avoid.
On 23 July 2018 at 13:19, Todd toddrjen@gmail.com wrote:
On Mon, Jul 23, 2018 at 7:24 AM, Paul Moore p.f.moore@gmail.com wrote:
On 23 July 2018 at 11:31, Jeroen Demeyer J.Demeyer@ugent.be wrote:
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to add additional syntax to the language for that. For example, one could say that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not a Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4] of slices too.
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
Paul
That involves initializing an instance, which doesn't serve any purpose in this case and I was hoping to avoid.
Well it serves the purpose that you can do it already in current Python, rather than needing a core interpreter change and limiting your code to Python 3.8+ only ;-)
Paul
On Mon, Jul 23, 2018 at 8:41 AM, Paul Moore p.f.moore@gmail.com wrote:
On 23 July 2018 at 13:19, Todd toddrjen@gmail.com wrote:
On Mon, Jul 23, 2018 at 7:24 AM, Paul Moore p.f.moore@gmail.com wrote:
On 23 July 2018 at 11:31, Jeroen Demeyer J.Demeyer@ugent.be wrote:
On 2018-07-23 12:24, Jeroen Demeyer wrote:
Another solution that nobody has mentioned (as far as I know) is to
add
additional syntax to the language for that. For example, one could
say
that (1:3) could be used to construct slice(1, 3) directly. The parentheses are required to avoid confusion with type hints. I'm not
a
Python language expert, but I don't think that type hints can occur inside parentheses like that.
And this could be extended to tuples (1:3, 2:4) and lists [1:3, 2:4]
of
slices too.
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
Paul
That involves initializing an instance, which doesn't serve any purpose
in
this case and I was hoping to avoid.
Well it serves the purpose that you can do it already in current Python, rather than needing a core interpreter change and limiting your code to Python 3.8+ only ;-)
Paul
Making it a classmethod would only require a metaclass, not a core interpreter change.
On Mon, Jul 23, 2018 at 4:24 AM Paul Moore p.f.moore@gmail.com wrote:
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
I think a SliceHelper class like this is a reasonable solution, but there would be a lot of value having it a standard place somewhere in the standard library (e.g., operator.subscript).
Both pandas and NumPy include this helper object under different names (pandas.IndexSlice and numpy.index_exp / numpy.s_), but it would be surprising/unexpected for pandas/numpy specific helpers to show up when not using one of those libraries. I do the exact same sorts of indexing manipulations with xarray, dask and TensorFlow.
Given that this is basically a simple feature to make it easier to work with Python syntax (so there's no danger it will change in the future), I think there is a lot to be said for putting it in the standard library in one place so it's obvious what to use and users don't have to relearn that name for this object and/or reimplement it.
Stephan Hoyer schrieb am 23.07.2018 um 18:01:
On Mon, Jul 23, 2018 at 4:24 AM Paul Moore wrote:
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
I think a SliceHelper class like this is a reasonable solution, but there would be a lot of value having it a standard place somewhere in the standard library (e.g., operator.subscript).
Both pandas and NumPy include this helper object under different names (pandas.IndexSlice and numpy.index_exp / numpy.s_), but it would be surprising/unexpected for pandas/numpy specific helpers to show up when not using one of those libraries. I do the exact same sorts of indexing manipulations with xarray, dask and TensorFlow.
Given that this is basically a simple feature to make it easier to work with Python syntax (so there's no danger it will change in the future), I think there is a lot to be said for putting it in the standard library in one place so it's obvious what to use and users don't have to relearn that name for this object and/or reimplement it.
Please copy that comment into the ticket and ask for it to be reopened.
https://bugs.python.org/issue24379
Stefan
I find pandas.IndexSlice makes a lot of operations easier to spell. As simple as it is, it's a valuable capability. Rather than every library—or a number of them anyway—creating the same 4 lines of code with a different name, it would be much nicer to have it as a class __getitem__ method, e.g. slice[...], or as an attribute of the slice object, such as slice.literal[...].
On Mon, Jul 23, 2018, 7:20 PM Stefan Behnel stefan_ml@behnel.de wrote:
Stephan Hoyer schrieb am 23.07.2018 um 18:01:
On Mon, Jul 23, 2018 at 4:24 AM Paul Moore wrote:
I thought the reason the proposal got nowhere was because it's pretty simple to define it yourself:
class SliceHelper:
... def __getitem__(self, slice): ... return slice ...
SH = SliceHelper() SH[1::3]
slice(1, None, 3)
Did I miss something significant about why this wasn't sufficient?
I think a SliceHelper class like this is a reasonable solution, but there would be a lot of value having it a standard place somewhere in the standard library (e.g., operator.subscript).
Both pandas and NumPy include this helper object under different names (pandas.IndexSlice and numpy.index_exp / numpy.s_), but it would be surprising/unexpected for pandas/numpy specific helpers to show up when
not
using one of those libraries. I do the exact same sorts of indexing manipulations with xarray, dask and TensorFlow.
Given that this is basically a simple feature to make it easier to work with Python syntax (so there's no danger it will change in the future), I think there is a lot to be said for putting it in the standard library in one place so it's obvious what to use and users don't have to relearn
that
name for this object and/or reimplement it.
Please copy that comment into the ticket and ask for it to be reopened.
https://bugs.python.org/issue24379
Stefan
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Jul 23, 2018 at 4:19 PM Stefan Behnel stefan_ml@behnel.de wrote:
Stephan Hoyer schrieb am 23.07.2018 um 18:01:
I think a SliceHelper class like this is a reasonable solution, but there would be a lot of value having it a standard place somewhere in the standard library (e.g., operator.subscript).
Both pandas and NumPy include this helper object under different names (pandas.IndexSlice and numpy.index_exp / numpy.s_), but it would be surprising/unexpected for pandas/numpy specific helpers to show up when
not
using one of those libraries. I do the exact same sorts of indexing manipulations with xarray, dask and TensorFlow.
Given that this is basically a simple feature to make it easier to work with Python syntax (so there's no danger it will change in the future), I think there is a lot to be said for putting it in the standard library in one place so it's obvious what to use and users don't have to relearn
that
name for this object and/or reimplement it.
Please copy that comment into the ticket and ask for it to be reopened.
https://bugs.python.org/issue24379
Stefan
I basically did exactly that last week! See https://bugs.python.org/issue24379#msg321966
I was told, "You may get more traction on python-ideas" :)
Cheers, Stephan
On Mon, Jul 23, 2018 at 4:37 PM David Mertz mertz@gnosis.cx wrote:
I find pandas.IndexSlice makes a lot of operations easier to spell. As simple as it is, it's a valuable capability. Rather than every library—or a number of them anyway—creating the same 4 lines of code with a different name, it would be much nicer to have it as a class __getitem__ method, e.g. slice[...], or as an attribute of the slice object, such as slice.literal[...].
I'd really like to move this proposal forward in some form.
There have been three basic proposals for where to put the __getitem__ based constructor: 1. slice: slice[...] potentially conflicts with typing syntax. 2. slice.literal: This is potentially confusing, because in many cases it does not actually create a single slice object. 3. operator.subscript: This is the last proposal standing, and was also the outcome of the previous discussion on python-ideas: https://mail.python.org/pipermail/python-ideas/2015-June/034086.html
I think the repeated interest in this topic demonstrates that there are real use cases for operator.subscript, even though it would be slightly less accessible than putting it directly on slice or slice.literal.
As a next step, would it be helpful to summarize the use cases and reasoning in a PEP?
Hi Stephan
I took a look at your personal web site. Your PhD sounds interesting. Quantum coherence and photosynthesis. But off-topic here. (Sadly, personal web sites are a bit of mess.)
You wrote:
I'd really like to move this proposal forward in some form.
+10
As a next step, would it be helpful to summarize the use cases and reasoning in a PEP?
0
Here's my two cents worth of opinion.
By all means start a PEP (on github) if you find it helps you. I think it's too early.
If this problem has a pure Python solution, then I think it best to develop it as a third party module. I suggest the name slicetools. When slicetools is established, has users and is stable. That would be the time to submit the PEP. I think it would give a smoother and easier path.
For example, PEP 428 proposes the inclusion of a third-party module, pathlib, in the standard library. https://mail.python.org/pipermail/python-ideas/2018-July/052255.html https://www.python.org/dev/peps/pep-0428/
For more on my general approach to building a third-party library, see my posts. https://mail.python.org/pipermail/python-ideas/2018-July/052464.html https://mail.python.org/pipermail/python-ideas/2018-July/052470.html
That's the end of my two cents.
Oh, and those who have time and energy to contribute. They get the biggest vote.
best regards
On Sat, Jul 28, 2018 at 9:44 AM Jonathan Fine jfine2358@gmail.com wrote:
By all means start a PEP (on github) if you find it helps you. I think it's too early.
If this problem has a pure Python solution, then I think it best to develop it as a third party module. I suggest the name slicetools. When slicetools is established, has users and is stable. That would be the time to submit the PEP. I think it would give a smoother and easier path.
For most proposals, I would agree that agree that a reference implementation as a third-party library makes sense. But operator.subscript so simple that it would be counter productive to write it in third-party module.
The pure Python reference implementation for operator.subscript is literally four lines (or three lines, with __class_getitem__ in Python 3.7):
class _Subscript: def __getitem__(self, key): return key subscript = _Subscript()
Very few people would use an implementation in a third-party package, because it seems unlikely to be worth the trouble of adding a dependency for so few lines of code -- copy & paste is actually more maintainable. This exact same operator also already exists in two popular third-party Python packages by the names numpy.s_ and pandas.IndexSlice.
That said, the actual logic is not obvious, unless you already understand the details of how Python's indexing works. So most user code does not benefit in clarity by copying and pasting it.
participants (8)
-
David Mertz -
Grégory Lielens -
Jeroen Demeyer -
Jonathan Fine -
Paul Moore -
Stefan Behnel -
Stephan Hoyer -
Todd