Steve Holden email@example.com wrote:
Fredrik Lundh wrote:
the problem isn't the time it takes to unpack the return value, the problem is that it takes time to create the substrings that you don't need.
Indeed, and therefore the performance of rpartition is likely to get worse as the length of the input strung increases. I don't like to think about all those strings being created just to be garbage-collected. Pity the poor CPU ... :-)
for some use cases, a naive partition-based solution is going to be a lot slower than the old find+slice approach, no matter how you slice, index, or unpack the return value.
Yup. Then it gets down to statistical arguments about the distribution of use cases and input lengths. If we had a type that represented a substring of an existing string it might avoid the stress, but I'm not sure I see that one flying.
What about buffer()? Tack on some string methods and you get a string slice-like instance with very low memory requirements. Add on actual comparisons of buffers and strings, and you can get nearly everything desired with very low memory overhead.
A bit of free thought brings me to the (half-baked) idea that if string methods accepted any object which conformed to the buffer interface; mmap, buffer, array, ... instances could gain all of the really convenient methods that make strings the objects to use in many cases.
If one wanted to keep string methods returning strings, and other objects with the buffer protocol which use string methods returning buffer objects, that seems reasonable (and probably a good idea).
P.S. Pardon me if the idea is pure insanity, I haven't been getting much sleep lately, and just got up from a nap that seems to have clouded my judgement (I just put milk in my juice...).