[Python-ideas] New explicit methods to trim strings
steve at pearwood.info
Mon Apr 1 20:52:52 EDT 2019
On Mon, Apr 01, 2019 at 02:29:44PM +1100, Chris Angelico wrote:
> The multiple affix case has exactly two forms:
> 1) Tearing multiple affixes off (eg stripping "asdf.jpg.png" down to
> just "asdf"), which most people are saying "no, don't do that, it
> doesn't make sense and isn't needed"
Perhaps I've missed something obvious (its been a long thread, and I'm
badly distracted with hardware issues that are causing me some
considerable grief), but I haven't seen anyone say "don't do that".
But I have seen David Mertz say that this was the best behaviour:
fname = 'silly.jpg.png.gif.png.jpg.gif.jpg'
I'm honestly not sure what behavior would be useful most often for
this oddball case. For the suffixes, I think "remove them all" is
probably the best
I'd also like to point out that this is not an oddball case. There are
two popular platforms where file extensions are advisory not mandatory
(Linux and Mac), but even on Windows it is possible to get files with
multiple, meaningful, extensions (foo.tar.gz for example) as well as
periods used in place of spaces (a.funny.cat.video.mp4).
> 2) Removing one of several options, which implies that one option is a
> strict subpiece of another (eg stripping off "test" and "st")
I take it you're only referring to the problematic cases, because
there's the third option, where none of the affixes to be removed clash:
But that's pretty uninteresting and a simple loop or repeated call to
the method will work fine:
just as we do with replace:
spam.replace(",", "").replace(" ", "")
If you only have a few affixes to work with, this is fine. If you have a
lot, you may want a helper function, but that's okay.
> If anyone is advocating for #1, I would agree with saying YAGNI.
David Mertz did.
> But #2 is an extremely unlikely edge case, and whatever semantics are
> chosen for it, *normal* usage will not be affected.
Not just unlikely, but "extremely" unlikely?
Presumably you didn't just pluck that statement out of thin air, but
have based it on an objective and statistically representative review of
existing code and projections of future uses of these new methods. How
could I possibly argue with that?
Except to say that I think it is recklessly irresponsible for people
engaged in language design to dismiss edge cases which will cause users
real bugs and real pain so easily. We're not designing for our personal
toolbox, we're designing for hundreds of thousands of other people with
widely varying needs.
It might be rare for you, but for somebody it will be happening ten
times a day. And for somebody else, it will only happen once a year, but
when it does, their code won't raise an exception it will just silently
do the wrong thing.
This is why replace does not take a set of multiple targets to replace.
The user, who knows their own use-case and what behaviour they want, can
write their own multiple-replace function, and we don't have to guess
what they want.
The point I am making is not that we must not ever support multiple
affixes, but that we shouldn't rush that decision. Let's pick the
low-hanging fruit, and get some real-world experience with the function
before deciding how to handle the multiple affix case.
> Or all the behaviours actually do the same thing anyway.
In this thread, I keep hearing this message:
"My own personal use-case will never be affected by clashing affixes, so
I don't care what behaviour we build into the language, so long as we
pick something RIGHT NOW and don't give the people actually affected
time to use the method and decide what works best in practice for them."
Like for the str.replace method, the final answer might be "there is no
best behaviour and we should refuse to choose".
Why are we rushing to permanently enshrine one specific behaviour into
the builtins before any of the users of the feature have a chance to use
it and decide for themselves which suits them best?
Now is better than never.
Although never is often better than *right* now.
Somebody (I won't name names, but they know who they are) wrote to me
off-list some time ago and accused me of being arrogant and thinking I
know more than everyone else. Well perhaps I am, but I'm not so arrogant
as to think that I can choose the right behaviour for clashing affixes
for other people when my own use-cases don't have clashing affixes.
> Sure, but I've often wanted to do something like "strip off a prefix
> of http:// or https://", or something else that doesn't have a
> semantic that's known to the stdlib.
I presume there's a reason you aren't using urllib.parse and you just
need a string without the leading scheme. If you're doing further
parsing, the stdlib has the right batteries for that.
(Aside: perhaps urllib.parse.ParseResult should get an attribute to
return the URL minus the scheme? That seems like it would be useful.)
> Also, this is still fairly
> verbose, and a lot of people are going to reach for a regex, just
> because it can be done in one line of code.
Okay, they will use a regex. Is this a problem? We're not planning on
banning regexes are we? If they're happy using regexes, and don't care
that it will be perhaps 3 times slower, let them.
> > I posted links to prior art. Unless I missed something, not one of those
> > languages or libraries supports multiple affixes in the one call.
> And they don't support multiple affixes in startswith/endswith either,
> but we're very happy to have that in Python.
But not until we had a couple of releases of experience with them:
And .replace still only takes a single target to be replaced.
> We don't have to worry about edge cases that are
> unlikely to come up in real-world code,
And you are making that pronouncement on the basis of what? Your gut
feeling? Perhaps you're thinking too narrowly.
Here's a partial list of English prefixes that somebody doing text
processing might want to remove to get at the root word:
a an ante anti auto circum co com con contra contro de dis
en ex extra hyper il im in ir inter intra intro macro micro
mono non omni post pre pro sub sym syn tele un uni up
I count fourteen clashes:
a: an ante anti
an: ante anti
co: com con contra contro
in: inter intra intro
(That's over a third of this admittedly incomplete list of prefixes.)
I can think of at least one English suffix pair that clash: -ify, -fy.
How about other languages? How comfortable are you to say that nobody
doing text processing in German or Hindi will need to deal with clashing
More information about the Python-ideas