Mailman 3 What are the strong use cases for str.rindex()? - Python-ideas

newer
Syntax to conditionally define a...

What are the strong use cases for str.rindex()?

older
Using rightarrow "->" for typing...

林自均

23 Apr 2019 23 Apr '19

6:44 a.m.

There were known, strong use cases for str.rindex(). The list.rindex() method was intentionally omitted. AFAICT no compelling use cases have arisen, so we should continue to leave it out. In general, we don't grow

Hi all, I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex(). So I filed the issue https://bugs.python.org/issue36639 to provide list.rindex(). However, the issue was rejected and closed with the comment: the core APIs unnecessarily. However, I am not sure what the known, strong use cases for str.rindex() are. Why doesn't the strong use cases apply on list.rindex()? Could anyone give me some examples? Thanks. Best, John Lin

Attachments:

attachment.htm (text/html — 979 bytes)

Show replies by date

Brett Cannon

23 Apr 23 Apr

5:28 p.m.

Given "abcdefabcdefabcdef", what is the last result of "abc"? x.rindex("abc") will tell you. Given [1, 2, 3, 4, 5, 1, 2, 3, 4, 5] where is the last result of 3? reversed(x).index(3) will tell you (or x[::-1]). Notice how with lists you can easily reverse them and still get at the value since you are searching per index. But with strings, you searching by a subslice that can be greater than 1 in which case you can't use a similar approach. On Mon, Apr 22, 2019 at 11:47 PM 林自均 wrote:

...

Hi all,

I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex(). So I filed the issue https://bugs.python.org/issue36639 to provide list.rindex(). However, the issue was rejected and closed with the comment:

...
There were known, strong use cases for str.rindex(). The list.rindex() method was intentionally omitted. AFAICT no compelling use cases have arisen, so we should continue to leave it out. In general, we don't grow the core APIs unnecessarily.

However, I am not sure what the known, strong use cases for str.rindex() are. Why doesn't the strong use cases apply on list.rindex()? Could anyone give me some examples? Thanks.

Best, John Lin _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano

25 Apr 25 Apr

12:07 a.m.

On Tue, Apr 23, 2019 at 10:28:29AM -0700, Brett Cannon wrote:

...

Given "abcdefabcdefabcdef", what is the last result of "abc"? x.rindex("abc") will tell you.

Given [1, 2, 3, 4, 5, 1, 2, 3, 4, 5] where is the last result of 3? reversed(x).index(3) will tell you (or x[::-1]).

That first version doesn't work, as list_reverseiterator objects don't have an index method. You're not the only person to make that error, I too often forget that reverse() returns an iterator, not a list. The second is easy to get wrong, because it returns the wrong index: # Get the item following the last instance of spam. index = x[::-1].index(spam) print(x[index+1]) In your example, the correct index is 7 but the returned value is 2.

...

Notice how with lists you can easily reverse them and still get at the value since you are searching per index.

"Easily" hides a lot of copying behind the scenes. If the list is a non-trivial size, that can be very wasteful, especially if you're doing it in a loop, or hidden in a function. Don't think about the case of a ten element list, think of a ten-billion element list. Personally, I don't think I've every used list.index, let alone needed rindex. But I think we underestimate the difficulty and cost of faking an rindex method from index for those who need it (if anyone does).

...

But with strings, you searching by a subslice that can be greater than 1 in which case you can't use a similar approach.

Of course you can: you "just" need to reverse the substring as well. The conversions will be even more fiddly and error-prone: py> s = "abc spam def spam ghi" py> s.rindex('spam') == len(s) - s[::-1].index('spam'[::-1]) - len('spam') True but it can be done. -- Steven

Greg Ewing

11:12 p.m.

Steven D'Aprano wrote:

...

I too often forget that reverse() returns an iterator,

That seems like a mistake. Shouldn't it return a view? -- Greg

Terry Reedy

26 Apr 26 Apr

4:17 a.m.

On 4/25/2019 7:12 PM, Greg Ewing wrote:

...

Steven D'Aprano wrote:

...
I too often forget that reverse() returns an iterator,

I presume you mean reversed(). list.reverse() is a list

...

That seems like a mistake. Shouldn't it return a view?

RL = reversed(somelist) is already partly view-like. The nth next call returns the nth item at the time of the next call, rather than at the time of the reversed call. However, the number of items produced by next calls is the length of the list at the time of the reversed call. The first next(RL) is the current somelist[captured_length].

...

...
...
somelist = [1,2,3] RL = reversed(somelist) somelist[-1] = None somelist.append(4) list(RL) [None, 2, 1]

-- Terry Jan Reedy

Steven D'Aprano

4:56 a.m.

On Fri, Apr 26, 2019 at 12:17:57AM -0400, Terry Reedy wrote:

...

On 4/25/2019 7:12 PM, Greg Ewing wrote:

...
Steven D'Aprano wrote:

...
I too often forget that reverse() returns an iterator,

I presume you mean reversed(). list.reverse() is a list

Yes, I meant reversed(), not list.reverse() which is an in-place mutator method and returns None.

...

...
That seems like a mistake. Shouldn't it return a view?

RL = reversed(somelist) is already partly view-like. The nth next call returns the nth item at the time of the next call, rather than at the time of the reversed call. However, the number of items produced by next calls is the length of the list at the time of the reversed call.

That's not quite correct: py> L = [1, 2, 3] py> R = reversed(L) py> L.clear() py> next(R) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration It seems that: - in-place modifications in the list are reflected in the items yielded (so reversed() doesn't make a copy of the list); - operations which extend the length of the list don't show up in the reversed version; - and operations which decrease the length of the list decrease the number of items yielded. That suggests to me an implementation similar to: # untested def reversed(alist): N = len(alist) for i in range(N-1, -1, -1): try: yield alist[i] except IndexError: break raise StopIteration which I suppose is close to what you meant here:

...

The first next(RL) is the current somelist[captured_length].

-- Steven

Steven D'Aprano

4:34 a.m.

On Fri, Apr 26, 2019 at 11:12:51AM +1200, Greg Ewing wrote:

...

Steven D'Aprano wrote:

...
I too often forget that reverse() returns an iterator,

That seems like a mistake. Shouldn't it return a view?

I don't know what it "should" or "shouldn't" it return, but it actually does return an iterator: py> L = [1, 2, 3] py> R = reversed(L) py> hasattr(R, '__iter__') and iter(R) is R True -- Steven

Terry Reedy

23 Apr 23 Apr

5:52 p.m.

On 4/23/2019 2:44 AM, 林自均 wrote:

...

Hi all,

I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex().

str.index and list.index are related but not the same. The consistency argument is better applied to find-rfind, index-rindex, partition-rpartition, etc. It is much more common to process strings right to left or in both directions, than process lists right to left or in both directions. Moreover, lists have a reverse method, strings do not. ''.join(reversed(somestring)) is likely slower, especially if there many non-ascii chars. Moreover, somestring.rindex(substring) would have to have both somestring and substring reversed when substring is more than one char. So I filed the issue

...

https://bugs.python.org/issue36639 to provide list.rindex(). However, the issue was rejected and closed with the comment:

...
There were known, strong use cases for str.rindex(). The list.rindex() method was intentionally omitted. AFAICT no compelling use cases have arisen, so we should continue to leave it out. In general, we don't grow the core APIs unnecessarily.

However, I am not sure what the known, strong use cases for str.rindex() are. Why doesn't the strong use cases apply on list.rindex()? Could anyone give me some examples? Thanks.

Omitting tests of rindex and rfind themselves: Searching 'rindex' in C:\Programs\Python38\lib\*.py ... C:\Programs\Python38\lib\_markupbase.py: 55: pos = rawdata.rindex("\n", i, j) # Should not fail C:\Programs\Python38\lib\collections\__init__.py: 1260: def rindex(self, sub, start=0, end=_sys.maxsize): C:\Programs\Python38\lib\collections\__init__.py: 1261: return self.data.rindex(sub, start, end) C:\Programs\Python38\lib\pydoc_data\topics.py: 10125: 'str.rindex(sub[, start[, end]])\n' C:\Programs\Python38\lib\test\pickletester.py: 2543: self.assertEqual((pickled.rindex(b"abcd") + len(b"abcd") - C:\Programs\Python38\lib\test\pickletester.py: 2574: self.assertEqual((pickled.rindex(b"abcd") + len(b"abcd") - C:\Programs\Python38\lib\test\test_baseexception.py: 46: depth = exc_line.rindex('-') C:\Programs\Python38\lib\test\test_bigmem.py: 310: self.assertEqual(s.rindex(_(' ')), C:\Programs\Python38\lib\test\test_bigmem.py: 311: sublen + size + SUBSTR.rindex(_(' '))) C:\Programs\Python38\lib\test\test_bigmem.py: 312: self.assertEqual(s.rindex(SUBSTR), sublen + size) C:\Programs\Python38\lib\test\test_bigmem.py: 313: self.assertEqual(s.rindex(_(' '), 0, sublen + size - 1), C:\Programs\Python38\lib\test\test_bigmem.py: 314: SUBSTR.rindex(_(' '))) C:\Programs\Python38\lib\test\test_bigmem.py: 315: self.assertEqual(s.rindex(SUBSTR, 0, sublen + size), 0) C:\Programs\Python38\lib\test\test_bigmem.py: 316: self.assertEqual(s.rindex(_('i')), C:\Programs\Python38\lib\test\test_bigmem.py: 317: sublen + size + SUBSTR.rindex(_('i'))) C:\Programs\Python38\lib\test\test_bigmem.py: 318: self.assertEqual(s.rindex(_('i'), 0, sublen), SUBSTR.rindex(_('i'))) C:\Programs\Python38\lib\test\test_bigmem.py: 319: self.assertEqual(s.rindex(_('i'), 0, sublen + size), C:\Programs\Python38\lib\test\test_bigmem.py: 320: SUBSTR.rindex(_('i'))) C:\Programs\Python38\lib\test\test_bigmem.py: 321: self.assertRaises(ValueError, s.rindex, _('j')) C:\Programs\Python38\lib\test\test_bytes.py: 566: self.assertEqual(b.rindex(b'ss'), 5) C:\Programs\Python38\lib\test\test_bytes.py: 567: self.assertRaises(ValueError, b.rindex, b'w') C:\Programs\Python38\lib\test\test_bytes.py: 568: self.assertRaises(ValueError, b.rindex, b'mississippian') C:\Programs\Python38\lib\test\test_bytes.py: 570: self.assertEqual(b.rindex(i), 10) C:\Programs\Python38\lib\test\test_bytes.py: 571: self.assertRaises(ValueError, b.rindex, w) C:\Programs\Python38\lib\test\test_bytes.py: 573: self.assertEqual(b.rindex(b'ss', 3), 5) C:\Programs\Python38\lib\test\test_bytes.py: 574: self.assertEqual(b.rindex(b'ss', 0, 6), 2) C:\Programs\Python38\lib\test\test_bytes.py: 576: self.assertEqual(b.rindex(i, 1, 3), 1) C:\Programs\Python38\lib\test\test_bytes.py: 577: self.assertEqual(b.rindex(i, 3, 9), 7) C:\Programs\Python38\lib\test\test_bytes.py: 578: self.assertRaises(ValueError, b.rindex, w, 1, 3) C:\Programs\Python38\lib\test\test_bytes.py: 768: self.assertEqual(3, b.rindex(l, None)) C:\Programs\Python38\lib\test\test_bytes.py: 769: self.assertEqual(3, b.rindex(l, -2, None)) C:\Programs\Python38\lib\test\test_bytes.py: 770: self.assertEqual(2, b.rindex(l, None, -2)) C:\Programs\Python38\lib\test\test_bytes.py: 771: self.assertEqual(0, b.rindex(h, None, None)) C:\Programs\Python38\lib\test\test_bytes.py: 791: for method in (b.count, b.find, b.index, b.rfind, b.rindex): C:\Programs\Python38\lib\test\test_bytes.py: 806: self.assertRaisesRegex(TypeError, r'\brindex\b', b.rindex, C:\Programs\Python38\lib\test\test_descr.py: 3596: try: ''.rindex('5') C:\Programs\Python38\lib\test\test_descr.py: 3598: else: self.fail("''.rindex('5') doesn't raise ValueError") Searching 'rfind' in C:\Programs\Python38\lib\*.py ... C:\Programs\Python38\lib\collections\__init__.py: 1256: def rfind(self, sub, start=0, end=_sys.maxsize): C:\Programs\Python38\lib\collections\__init__.py: 1259: return self.data.rfind(sub, start, end) C:\Programs\Python38\lib\ctypes\macholib\dyld.py: 143: fmwk_index = fn.rfind('.framework') C:\Programs\Python38\lib\doctest.py: 341: i = msg.rfind('.', 0, end) C:\Programs\Python38\lib\email\_parseaddr.py: 76: i = data[0].rfind(',') C:\Programs\Python38\lib\encodings\punycode.py: 187: pos = text.rfind(b"-") C:\Programs\Python38\lib\formatter.py: 401: i = data.rfind('\n') C:\Programs\Python38\lib\genericpath.py: 124: sepIndex = p.rfind(sep) C:\Programs\Python38\lib\genericpath.py: 126: altsepIndex = p.rfind(altsep) C:\Programs\Python38\lib\genericpath.py: 129: dotIndex = p.rfind(extsep) C:\Programs\Python38\lib\html\parser.py: 148: amppos = rawdata.rfind('&', max(i, n-34)) C:\Programs\Python38\lib\html\parser.py: 336: - self.__starttag_text.rfind("\n") C:\Programs\Python38\lib\http\client.py: 873: i = host.rfind(':') C:\Programs\Python38\lib\http\client.py: 874: j = host.rfind(']') # ipv6 addresses have [...] C:\Programs\Python38\lib\http\cookiejar.py: 566: i = A.rfind(B) C:\Programs\Python38\lib\http\cookiejar.py: 1016: i = domain.rfind(".") C:\Programs\Python38\lib\http\cookiejar.py: 1017: j = domain.rfind(".", 0, i) C:\Programs\Python38\lib\http\cookiejar.py: 1507: i = path.rfind("/") C:\Programs\Python38\lib\idlelib\pyparse.py: 161: i = code.rfind(":\n", 0, limit) C:\Programs\Python38\lib\idlelib\pyparse.py: 164: i = code.rfind('\n', 0, i) + 1 # start of colon line (-1+1=0) C:\Programs\Python38\lib\idlelib\pyparse.py: 379: p = code.rfind('\n', 0, p-1) + 1 C:\Programs\Python38\lib\idlelib\pyparse.py: 477: origi = i = code.rfind('\n', 0, j) + 1 C:\Programs\Python38\lib\json\decoder.py: 33: colno = pos - doc.rfind('\n', 0, pos) C:\Programs\Python38\lib\logging\__init__.py: 1340: i = name.rfind(".") C:\Programs\Python38\lib\logging\__init__.py: 1353: i = name.rfind(".", 0, i - 1) C:\Programs\Python38\lib\modulefinder.py: 155: i = pname.rfind('.') C:\Programs\Python38\lib\modulefinder.py: 509: i = name.rfind(".") C:\Programs\Python38\lib\pathlib.py: 791: i = name.rfind('.') C:\Programs\Python38\lib\pathlib.py: 810: i = name.rfind('.') C:\Programs\Python38\lib\pdb.py: 629: colon = arg.rfind(':') C:\Programs\Python38\lib\pdb.py: 884: i = arg.rfind(':') C:\Programs\Python38\lib\posixpath.py: 109: i = p.rfind(sep) + 1 C:\Programs\Python38\lib\posixpath.py: 148: i = p.rfind(sep) + 1 C:\Programs\Python38\lib\posixpath.py: 158: i = p.rfind(sep) + 1 C:\Programs\Python38\lib\pyclbr.py: 145: i = module.rfind('.') C:\Programs\Python38\lib\pydoc.py: 1657: desc += ' in ' + name[:name.rfind('.')] C:\Programs\Python38\lib\pydoc_data\topics.py: 10115: 'str.rfind(sub[, start[, end]])\n' C:\Programs\Python38\lib\pydoc_data\topics.py: 10127: ' Like "rfind()" but raises "ValueError" when the ' C:\Programs\Python38\lib\site-packages\pip\_vendor\html5lib\_inputstream.py: 228: lastLinePos = chunk.rfind('\n', 0, offset) C:\Programs\Python38\lib\site-packages\pip\_vendor\pyparsing.py: 1110: return 1 if 0

MRAB

7:56 p.m.

On 2019-04-23 18:52, Terry Reedy wrote:

...

On 4/23/2019 2:44 AM, 林自均 wrote:

...
Hi all,

I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex().

str.index and list.index are related but not the same. The consistency argument is better applied to find-rfind, index-rindex, partition-rpartition, etc.

It is much more common to process strings right to left or in both directions, than process lists right to left or in both directions. Moreover, lists have a reverse method, strings do not. ''.join(reversed(somestring)) is likely slower, especially if there many non-ascii chars. Moreover, somestring.rindex(substring) would have to have both somestring and substring reversed when substring is more than one char.

You can reverse a string with somestring[::-1]. Personally, I'm not convinced by the "lists can be reversed" argument. [snip]

Guido van Rossum

8:18 p.m.

On Tue, Apr 23, 2019 at 1:02 PM MRAB wrote:

...

On 2019-04-23 18:52, Terry Reedy wrote:

...
On 4/23/2019 2:44 AM, 林自均 wrote:

...
Hi all,

I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex().

str.index and list.index are related but not the same. The consistency argument is better applied to find-rfind, index-rindex, partition-rpartition, etc.

It is much more common to process strings right to left or in both directions, than process lists right to left or in both directions. Moreover, lists have a reverse method, strings do not. ''.join(reversed(somestring)) is likely slower, especially if there many non-ascii chars. Moreover, somestring.rindex(substring) would have to have both somestring and substring reversed when substring is more than one char.

You can reverse a string with somestring[::-1].

Personally, I'm not convinced by the "lists can be reversed" argument.

Me neither, though for substring checks, reversing the string would be even more cumbersome (you'd have to reverse the query string too). My money is on "nobody uses this for lists". Some use cases for rindex() on strings that I found in a large codebase here include searching a pathname for the final slash, a list of comma-separated items for the last comma, a fully-qualified module name for the last period, and some ad-hoc parsing of other things. The "last separator" use cases are the most common and here rindex() sounds very useful. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

林自均

24 Apr 24 Apr

12:59 a.m.

Hi all, Thanks for the explanation. Now I agree that the need for list.rindex() is not as common as str.rindex(). In fact, I only need list.rindex() when doing some algorithm problems. I guess that doesn't count as real need here. Best, John Lin Guido van Rossum 於 2019年4月24日週三上午4:20寫道：

...

On Tue, Apr 23, 2019 at 1:02 PM MRAB wrote:

...
On 2019-04-23 18:52, Terry Reedy wrote:

...
On 4/23/2019 2:44 AM, 林自均 wrote:

...
Hi all,

I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex().

str.index and list.index are related but not the same. The consistency argument is better applied to find-rfind, index-rindex, partition-rpartition, etc.

It is much more common to process strings right to left or in both directions, than process lists right to left or in both directions. Moreover, lists have a reverse method, strings do not. ''.join(reversed(somestring)) is likely slower, especially if there many non-ascii chars. Moreover, somestring.rindex(substring) would have to have both somestring and substring reversed when substring is more than one char.

You can reverse a string with somestring[::-1].

Personally, I'm not convinced by the "lists can be reversed" argument.

Me neither, though for substring checks, reversing the string would be even more cumbersome (you'd have to reverse the query string too).

My money is on "nobody uses this for lists".

Some use cases for rindex() on strings that I found in a large codebase here include searching a pathname for the final slash, a list of comma-separated items for the last comma, a fully-qualified module name for the last period, and some ad-hoc parsing of other things. The "last separator" use cases are the most common and here rindex() sounds very useful.

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano

25 Apr 25 Apr

12:11 a.m.

On Wed, Apr 24, 2019 at 08:59:18AM +0800, 林自均 wrote:

...

Hi all,

Thanks for the explanation. Now I agree that the need for list.rindex() is not as common as str.rindex(). In fact, I only need list.rindex() when doing some algorithm problems. I guess that doesn't count as real need here.

Of course it's a "real need", but the question is whether it is common enough for somebody to do the work if it doesn't affect them personally. Can you share an example of one of these algorithms? -- Steven

1821

Age (days ago)

1824

Last active (days ago)

List overview

Download

11 comments

7 participants

participants (7)

Brett Cannon
Greg Ewing
Guido van Rossum
MRAB
Steven D'Aprano
Terry Reedy
林自均

What are the strong use cases for str.rindex()?

tags

participants (7)