str.strip: argument maxstrip
I often find myself in a need of stripping only a specified amount of characters from a string (most commonly a single character). I have been implementing this in an ad-hoc manner, which is quite inelegant: def rstrip1(txt, chars): if txt is None: return None elif any(txt.endswith(c) for c in chars): return txt[:-1] else: return txt I would appreciate if str.split, str.lstrip, str.rsplit functions had a `maxstrip` argument, similar to the `maxsplit` argument of `str.split`, which would specify the maximum count of characters to be removed from the string. In case of `str.split` this maximum would apply to each side of the string separately. Am I the only one who is surprised such functionality is not implemented already??
To strip at most 1 character from the end: txt[:-1] + txt[-1:].rstrip(chars) To strip at most N characters: txt[:-N] + txt[-N:].rstrip(chars) Given that, I think yours is too much of a niche case to change anything in Python Rob Cliffe On 19/05/2020 12:44, computermaster360 . wrote:
I think this would be useful, and doesn't break any backward compatibility. I would have use for it from time to time myself. On Tue, May 19, 2020 at 9:01 AM computermaster360 . < computermaster360@gmail.com> wrote:
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David (or somebody else) could you give us some, as real as possible, examples? This will strengthen the case for it! I am confident they exist and are pretty plentiful but I myself am coming up blank thinking about it for a few minutes and documenting them would be good for discussion. On Tue, 19 May 2020 at 18:59, David Mertz <mertz@gnosis.cx> wrote:
I did a couple greps of my git/ directory to see if I found examples. Given that there are a couple ways one might achieve the effect now, I don't necessarily find everything. But here's something in software I did not write myself. This is from ./JohnTheRipper/run/sspr2john.py. I found another example in a different file, but looking at it I'm pretty sure it is actually a potential bug (it has a comment similar to "Is this safe?" which I won't bother showing. elif fmt == "PBKDF2_SHA256": h = base64.b64encode(base64.b64decode(text)[:32]) # a terrible hack follows, use "adapted base64" alphabet (using . instead of + and with no padding) h = h.rstrip("=").replace("+", ".") salt = base64.b64encode(salt) salt = salt.rstrip("=").replace("+", ".") We actually know that base64 code should only produce at most 2 '='s as padding. In this instance, the encoding comes immediately before the stripping. However, perhaps some code would pass the encoded string and you wouldn't be as confident locally that extra '='s hadn't snuck in. If it existed, I think these lines would be good candidates for 'maxstrip'. On Tue, May 19, 2020 at 2:07 PM Henk-Jaap Wagenaar < wagenaarhenkjaap@gmail.com> wrote:
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Tue, May 19, 2020 at 8:49 PM David Mertz <mertz@gnosis.cx> wrote:
Not a very strong ending 🤣 I may be misunderstanding, but it sounds like = is not acceptable in the final result, so it's not enough to remove only 2 of 4 ='s. You want to make sure nothing messed up your string. So if the code existed, what you'd want is: ``` assert salt.count("=") <= 2 salt = salt.rstrip("=", "") assert "=" not in salt ```
I think the code I'd want, if the new parameter existed, would be: salt = salt.rstrip("=", maxstrip=2) assert not salt.endswith("=") That feels *slightly* better than your version. This version would allow there to be '=' in the middle of the string (albeit, such would not be base64, but some other kind of thing). Obviously, I've managed to program Python without this switch for 20+ years. But I think I'd use it occasionally. Currently, I'd probably program my intention like this. Let's assume this is something where the '=' is allowed in the middle. if salt.endswith("=="): salt = salt[-2:] elif salt.endswith("="): salt = salt[-1:] assert not salt.endswith("=") The version you suggested, as mentioned, would not handle a format where '=' was permitted in the middle. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Tue, May 19, 2020 at 9:43 PM David Mertz <mertz@gnosis.cx> wrote:
Currently, I'd probably program my intention like this. Let's assume this is something where the '=' is allowed in the middle.
Getting increasingly hypothetical... Anyway, you could write: ``` assert not salt.endswith("=" * 3) # at most 2 ='s allowed salt = salt.rstrip("=") ```
On Tue, May 19, 2020 at 3:50 PM Alex Hall <alex.mojaki@gmail.com> wrote:
Yes, I *could* write that. It feels a bit more cryptic than the version I mentioned. But it's fine. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On 19May2020 15:43, David Mertz <mertz@gnosis.cx> wrote:
Reiterating the Python 3.9 suggestion, what about: salt2 = salt.cutsuffix(('==', '=')) I appreciate this isn't as _general_ as a maxstrip param, but it seems to neatly address the single use case you've found. Cheers, Cameron Simpson <cs@cskk.id.au>
On Tue, May 19, 2020 at 8:43 PM Cameron Simpson <cs@cskk.id.au> wrote:
You'd have to call cutsuffix twice. Valid base64 might end in one, two, or zero '=', but the result wants to have none. Actually, no... even two calls won't always do the right thing. I'm happy to have the new method .cutsuffix() , but I don't think it addresses this case. I was going to suggest an example about dynamically built path components that may or may not end in '/'. However: A. I couldn't find an example in my own code easily, even though I know I've fiddled with that B. Someone would scold me for playing with strings rather than using Pathlib (they'd be right, I realize... but I am sinful in my quick-script ways). I'll let someone else, like the OP, advance the case more. I kinda like it, but it's not a big deal for me. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Wed, May 20, 2020 at 2:46 AM Cameron Simpson <cs@cskk.id.au> wrote:
Tuple arguments were rejected in the PEP. ``` Python 3.10.0a0 (heads/master:bac170cd93, May 20 2020, 12:20:34) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information.
But if the API was there, I agree this would work, not sure what David is
saying about needing to call twice. On the other hand, this example
demonstrates well how a tuple is potentially confusing. What happens if you
call `'abc=='.removesuffix(('=', '=='))`?
On Tue, May 19, 2020 at 2:58 PM computermaster360 . < computermaster360@gmail.com> wrote:
If you want to remove at most one instance of one specific character, then I believe in Python 3.9 you will be able to write `txt.removesuffix(char)` - see https://www.python.org/dev/peps/pep-0616/. For removing one of several possible characters, the potential API for that has currently been rejected: https://www.python.org/dev/peps/pep-0616/#accepting-a-tuple-of-affixes
To strip at most 1 character from the end: txt[:-1] + txt[-1:].rstrip(chars) To strip at most N characters: txt[:-N] + txt[-N:].rstrip(chars) Given that, I think yours is too much of a niche case to change anything in Python Rob Cliffe On 19/05/2020 12:44, computermaster360 . wrote:
I think this would be useful, and doesn't break any backward compatibility. I would have use for it from time to time myself. On Tue, May 19, 2020 at 9:01 AM computermaster360 . < computermaster360@gmail.com> wrote:
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David (or somebody else) could you give us some, as real as possible, examples? This will strengthen the case for it! I am confident they exist and are pretty plentiful but I myself am coming up blank thinking about it for a few minutes and documenting them would be good for discussion. On Tue, 19 May 2020 at 18:59, David Mertz <mertz@gnosis.cx> wrote:
I did a couple greps of my git/ directory to see if I found examples. Given that there are a couple ways one might achieve the effect now, I don't necessarily find everything. But here's something in software I did not write myself. This is from ./JohnTheRipper/run/sspr2john.py. I found another example in a different file, but looking at it I'm pretty sure it is actually a potential bug (it has a comment similar to "Is this safe?" which I won't bother showing. elif fmt == "PBKDF2_SHA256": h = base64.b64encode(base64.b64decode(text)[:32]) # a terrible hack follows, use "adapted base64" alphabet (using . instead of + and with no padding) h = h.rstrip("=").replace("+", ".") salt = base64.b64encode(salt) salt = salt.rstrip("=").replace("+", ".") We actually know that base64 code should only produce at most 2 '='s as padding. In this instance, the encoding comes immediately before the stripping. However, perhaps some code would pass the encoded string and you wouldn't be as confident locally that extra '='s hadn't snuck in. If it existed, I think these lines would be good candidates for 'maxstrip'. On Tue, May 19, 2020 at 2:07 PM Henk-Jaap Wagenaar < wagenaarhenkjaap@gmail.com> wrote:
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Tue, May 19, 2020 at 8:49 PM David Mertz <mertz@gnosis.cx> wrote:
Not a very strong ending 🤣 I may be misunderstanding, but it sounds like = is not acceptable in the final result, so it's not enough to remove only 2 of 4 ='s. You want to make sure nothing messed up your string. So if the code existed, what you'd want is: ``` assert salt.count("=") <= 2 salt = salt.rstrip("=", "") assert "=" not in salt ```
I think the code I'd want, if the new parameter existed, would be: salt = salt.rstrip("=", maxstrip=2) assert not salt.endswith("=") That feels *slightly* better than your version. This version would allow there to be '=' in the middle of the string (albeit, such would not be base64, but some other kind of thing). Obviously, I've managed to program Python without this switch for 20+ years. But I think I'd use it occasionally. Currently, I'd probably program my intention like this. Let's assume this is something where the '=' is allowed in the middle. if salt.endswith("=="): salt = salt[-2:] elif salt.endswith("="): salt = salt[-1:] assert not salt.endswith("=") The version you suggested, as mentioned, would not handle a format where '=' was permitted in the middle. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Tue, May 19, 2020 at 9:43 PM David Mertz <mertz@gnosis.cx> wrote:
Currently, I'd probably program my intention like this. Let's assume this is something where the '=' is allowed in the middle.
Getting increasingly hypothetical... Anyway, you could write: ``` assert not salt.endswith("=" * 3) # at most 2 ='s allowed salt = salt.rstrip("=") ```
On Tue, May 19, 2020 at 3:50 PM Alex Hall <alex.mojaki@gmail.com> wrote:
Yes, I *could* write that. It feels a bit more cryptic than the version I mentioned. But it's fine. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On 19May2020 15:43, David Mertz <mertz@gnosis.cx> wrote:
Reiterating the Python 3.9 suggestion, what about: salt2 = salt.cutsuffix(('==', '=')) I appreciate this isn't as _general_ as a maxstrip param, but it seems to neatly address the single use case you've found. Cheers, Cameron Simpson <cs@cskk.id.au>
On Tue, May 19, 2020 at 8:43 PM Cameron Simpson <cs@cskk.id.au> wrote:
You'd have to call cutsuffix twice. Valid base64 might end in one, two, or zero '=', but the result wants to have none. Actually, no... even two calls won't always do the right thing. I'm happy to have the new method .cutsuffix() , but I don't think it addresses this case. I was going to suggest an example about dynamically built path components that may or may not end in '/'. However: A. I couldn't find an example in my own code easily, even though I know I've fiddled with that B. Someone would scold me for playing with strings rather than using Pathlib (they'd be right, I realize... but I am sinful in my quick-script ways). I'll let someone else, like the OP, advance the case more. I kinda like it, but it's not a big deal for me. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Wed, May 20, 2020 at 2:46 AM Cameron Simpson <cs@cskk.id.au> wrote:
Tuple arguments were rejected in the PEP. ``` Python 3.10.0a0 (heads/master:bac170cd93, May 20 2020, 12:20:34) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information.
But if the API was there, I agree this would work, not sure what David is
saying about needing to call twice. On the other hand, this example
demonstrates well how a tuple is potentially confusing. What happens if you
call `'abc=='.removesuffix(('=', '=='))`?
On Tue, May 19, 2020 at 2:58 PM computermaster360 . < computermaster360@gmail.com> wrote:
If you want to remove at most one instance of one specific character, then I believe in Python 3.9 you will be able to write `txt.removesuffix(char)` - see https://www.python.org/dev/peps/pep-0616/. For removing one of several possible characters, the potential API for that has currently been rejected: https://www.python.org/dev/peps/pep-0616/#accepting-a-tuple-of-affixes
participants (7)
-
Alex Hall -
Cameron Simpson -
computermaster360 . -
David Mertz -
Henk-Jaap Wagenaar -
Jonathan Goble -
Rob Cliffe