Hi. I am working on a kinda-ORM library, which usage often implies to request data within specific ranges: Foobar.search( attr1="foo", attr2=gt(10), attr3=between(42, 50) ) The use of "gt" or "between" methods to describe those operations feels a bit cumbersome (as it is long to write, and you need to import those functions in a lot of files), and I though it could be more pythonic to use slices instead. Foobar.search( attr1="foo", attr2=slice(10), attr3=slice(42, 50) ) I suggest an alternative way to instanciate slices, that would look like this: Foobar.search( attr1="foo", attr2=[10:], attr3=[42:50] ) What do you think?
An alternative suggestion, which works today (... is a valid object called Ellipsis): Foobar.search( attr1="foo", attr2=[10, ...], attr3=[42, ..., 50] ) On Thu, Aug 12, 2021 at 8:44 AM <eloi.rivard@aquilenet.fr> wrote:
-- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman@redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
On Thu, Aug 12, 2021 at 9:02 AM Calvin Spealman <cspealma@redhat.com> wrote:
This got me thinking just now: allowing ellipses instead of None for the first two arguments of the slice() constructor might be a neat idea. These looks pretty nice: slice(1, ...) slice(1, ..., 2) slice(-1, ..., -1) slice(..., ..., 3) I wonder if-- had the ellipses existed when slice() was created eons ago/in the depths of time-- whether ... would have been used rather than None for these. I'm not really proposing this, just musing. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Thu, Aug 12, 2021 at 09:57:06AM -0400, Ricky Teachey wrote:
This got me thinking just now: allowing ellipses instead of None for the first two arguments of the slice() constructor might be a neat idea.
Like this?
The arguments to slice can be anything you like.
slice({'a': object()}, int, ('spam', 'eggs')) slice({'a': <object object at 0x7f0067ed9420>}, <class 'int'>, ('spam', 'eggs'))
and the interpretation is entirely up to the class:
Aside from the use of a literal `...` for Ellipsis, which is a lot more recent, this behaviour goes back to Python 1.5 or older. Slices have always been liberal about what they accept. -- Steve
On Thu, Aug 12, 2021 at 5:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
Oh man thanks for pointing that out... and of course numpy and pandas and many other libraries already use ellipses for various things in slices. and of course native python sequences already have far better syntactic sugar than ellipses: [][3:] [][::-1] etc etc. So given that, why would we ever want to use ellipses?!?! --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On 12Aug2021 17:45, Ricky Teachey <ricky@teachey.org> wrote:
I've got an I/O module where the take(n) call accepts an Ellipsis to mean "everything to the end of input". I know Python file.read() without a parameter also means that, but the idiom of my module uses take(n) to obtain exactly n bytes, so the Ellipsis is a good fit. Of course expressed as "...". Cheers, Cameron Simpson <cs@cskk.id.au>
Definitively not as short as what I suggested, but I love it. I think this is one of the cleanest way I can do right now. Thanks!
On 2021-08-12 at 08:56:22 -0000, eloi.rivard@aquilenet.fr wrote:
I think there will be a lot of off-by-1 errors. BETWEEN is inclusive on both ends. BETWEEN 42 AND 50 includes records where the value is 50. Python slices are open ended intervals. slice(42, 50) would *not* include the value 50. That tells me that slices are *not* a good substitute for BETWEEN when it comes to database queries.
This is how slices are used in the python standard library, indeed, but that does not stop me from interpreting the slices as "inclusive by default" in my library. The inconsistency with the rest of the python standard library could be misleading, but I think maybe less than off-by-1 errors? You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
On 2021-08-12 at 14:05:23 -0000, eloi.rivard@aquilenet.fr wrote:
https://docs.python.org/3/library/functions.html#slice says that slice objects represent "the set of indices specified by range(start, stop, step)." You are free to interpret a slice differently, but you may be in for a fight against (or bug reports from) existing Pythonistas. Also, given that slices explicitly represent indices, your API may be clearer if you used ranges instead of slices, but the question of what the end is would remain. YMMV.
You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
By definition, slices derive from ranges, and by definition, ranges do not include their ends. A (stop, stop) pair that includes the stop value is not a Python slice or a Python range. If you're willing to write your own gt class or function, then why not bt? That said, I don't think I've ever explicitly created a slice. And the better iteration works, the less I use range. I also can't recall using BETWEEN in database queries (except maybe for dates?), but databases and database queries are *not* my thing. In that light, consider me -0 on any new syntax for creating slices.
On 12Aug2021 14:05, eloi.rivard@aquilenet.fr <eloi.rivard@aquilenet.fr> wrote:
This is how slices are used in the python standard library, indeed, but that does not stop me from interpreting the slices as "inclusive by default" in my library. The inconsistency with the rest of the python standard library could be misleading, but I think maybe less than off-by-1 errors?
I think the misleadingness is identical to off-by-1 errors.
You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
Well, I have seen this done using a combination of square and round brackets: [12,19) which, IIRC, means inclusive on the left end (from the "[") and exclusive on the right (from the ")"). Obviously this would be a disaster in Python code, alas. I do like the idea of bare slices but I also almost never make them. The is perhaps because they're tedious to type. Instead I find myself doing one of 2 things: - a function f(low,high) returning something to do with that range - a instance method __getitem__ handling slices, so the user can write x[12:15] directly. Not creating a slice directly, but getting that range from "x", a sliceable object Have you looked at SQLAlchemy's core SQL stuff? Tables have columns, so you can write: the_table.c.column_name to reference the column "column_name". That is actually a reference to a column. It lets you write: the_table.c.column_name >= 12 and the_table.c.column_name < 15 in code and get SQL out the end. Of course the concise way is sometimes col = the_table.c.column_name col >= 12 and col < 15 You can probably even write: 12 <= col < 15 using the normal Python syntax. This works because a column reference implemenets __lt__ and friends, so that Python comparisons return "SQL comparison objects". You could implement such objects too. And have then support slicing via __getitem__. That might return some kind of "range of column values" objects, which could be used in expressions like this: attr[12:15] Unfortunately that looks to my eye like "get me these elements" rather than a test, but in the right context (your queries) it might be intuitive. Cheers, Cameron Simpson <cs@cskk.id.au>
In addition to the other answers: Libraries like NumPy define a helper factory for slices, e.g. ```python
You can do the same in your library. Equally, you don't have to return a slice — you might want to return a "Range" object as mentioned by other posters.
Personally, I don't think there needs to be a more concise syntax — people rarely need bare slice objects by themselves.
An alternative suggestion, which works today (... is a valid object called Ellipsis): Foobar.search( attr1="foo", attr2=[10, ...], attr3=[42, ..., 50] ) On Thu, Aug 12, 2021 at 8:44 AM <eloi.rivard@aquilenet.fr> wrote:
-- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman@redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
On Thu, Aug 12, 2021 at 9:02 AM Calvin Spealman <cspealma@redhat.com> wrote:
This got me thinking just now: allowing ellipses instead of None for the first two arguments of the slice() constructor might be a neat idea. These looks pretty nice: slice(1, ...) slice(1, ..., 2) slice(-1, ..., -1) slice(..., ..., 3) I wonder if-- had the ellipses existed when slice() was created eons ago/in the depths of time-- whether ... would have been used rather than None for these. I'm not really proposing this, just musing. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Thu, Aug 12, 2021 at 09:57:06AM -0400, Ricky Teachey wrote:
This got me thinking just now: allowing ellipses instead of None for the first two arguments of the slice() constructor might be a neat idea.
Like this?
The arguments to slice can be anything you like.
slice({'a': object()}, int, ('spam', 'eggs')) slice({'a': <object object at 0x7f0067ed9420>}, <class 'int'>, ('spam', 'eggs'))
and the interpretation is entirely up to the class:
Aside from the use of a literal `...` for Ellipsis, which is a lot more recent, this behaviour goes back to Python 1.5 or older. Slices have always been liberal about what they accept. -- Steve
On Thu, Aug 12, 2021 at 5:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
Oh man thanks for pointing that out... and of course numpy and pandas and many other libraries already use ellipses for various things in slices. and of course native python sequences already have far better syntactic sugar than ellipses: [][3:] [][::-1] etc etc. So given that, why would we ever want to use ellipses?!?! --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On 12Aug2021 17:45, Ricky Teachey <ricky@teachey.org> wrote:
I've got an I/O module where the take(n) call accepts an Ellipsis to mean "everything to the end of input". I know Python file.read() without a parameter also means that, but the idiom of my module uses take(n) to obtain exactly n bytes, so the Ellipsis is a good fit. Of course expressed as "...". Cheers, Cameron Simpson <cs@cskk.id.au>
Definitively not as short as what I suggested, but I love it. I think this is one of the cleanest way I can do right now. Thanks!
On 2021-08-12 at 08:56:22 -0000, eloi.rivard@aquilenet.fr wrote:
I think there will be a lot of off-by-1 errors. BETWEEN is inclusive on both ends. BETWEEN 42 AND 50 includes records where the value is 50. Python slices are open ended intervals. slice(42, 50) would *not* include the value 50. That tells me that slices are *not* a good substitute for BETWEEN when it comes to database queries.
This is how slices are used in the python standard library, indeed, but that does not stop me from interpreting the slices as "inclusive by default" in my library. The inconsistency with the rest of the python standard library could be misleading, but I think maybe less than off-by-1 errors? You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
On 2021-08-12 at 14:05:23 -0000, eloi.rivard@aquilenet.fr wrote:
https://docs.python.org/3/library/functions.html#slice says that slice objects represent "the set of indices specified by range(start, stop, step)." You are free to interpret a slice differently, but you may be in for a fight against (or bug reports from) existing Pythonistas. Also, given that slices explicitly represent indices, your API may be clearer if you used ranges instead of slices, but the question of what the end is would remain. YMMV.
You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
By definition, slices derive from ranges, and by definition, ranges do not include their ends. A (stop, stop) pair that includes the stop value is not a Python slice or a Python range. If you're willing to write your own gt class or function, then why not bt? That said, I don't think I've ever explicitly created a slice. And the better iteration works, the less I use range. I also can't recall using BETWEEN in database queries (except maybe for dates?), but databases and database queries are *not* my thing. In that light, consider me -0 on any new syntax for creating slices.
On 12Aug2021 14:05, eloi.rivard@aquilenet.fr <eloi.rivard@aquilenet.fr> wrote:
This is how slices are used in the python standard library, indeed, but that does not stop me from interpreting the slices as "inclusive by default" in my library. The inconsistency with the rest of the python standard library could be misleading, but I think maybe less than off-by-1 errors?
I think the misleadingness is identical to off-by-1 errors.
You raise a good point however, that is: how to write a slice with expliciting the inclusiveness of one of the limit values?
Well, I have seen this done using a combination of square and round brackets: [12,19) which, IIRC, means inclusive on the left end (from the "[") and exclusive on the right (from the ")"). Obviously this would be a disaster in Python code, alas. I do like the idea of bare slices but I also almost never make them. The is perhaps because they're tedious to type. Instead I find myself doing one of 2 things: - a function f(low,high) returning something to do with that range - a instance method __getitem__ handling slices, so the user can write x[12:15] directly. Not creating a slice directly, but getting that range from "x", a sliceable object Have you looked at SQLAlchemy's core SQL stuff? Tables have columns, so you can write: the_table.c.column_name to reference the column "column_name". That is actually a reference to a column. It lets you write: the_table.c.column_name >= 12 and the_table.c.column_name < 15 in code and get SQL out the end. Of course the concise way is sometimes col = the_table.c.column_name col >= 12 and col < 15 You can probably even write: 12 <= col < 15 using the normal Python syntax. This works because a column reference implemenets __lt__ and friends, so that Python comparisons return "SQL comparison objects". You could implement such objects too. And have then support slicing via __getitem__. That might return some kind of "range of column values" objects, which could be used in expressions like this: attr[12:15] Unfortunately that looks to my eye like "get me these elements" rather than a test, but in the right context (your queries) it might be intuitive. Cheers, Cameron Simpson <cs@cskk.id.au>
In addition to the other answers: Libraries like NumPy define a helper factory for slices, e.g. ```python
You can do the same in your library. Equally, you don't have to return a slice — you might want to return a "Range" object as mentioned by other posters.
Personally, I don't think there needs to be a more concise syntax — people rarely need bare slice objects by themselves.
participants (8)
-
2QdxY4RzWzUUiLuE＠potatochowder.com -
Angus Hollands -
Calvin Spealman -
Cameron Simpson -
Chris Angelico -
eloi.rivard＠aquilenet.fr -
Ricky Teachey -
Steven D'Aprano