[Python-Dev] Proposal to revert r54204 (splitext change)

["Martin v. Löwis"]

Phillip J. Eby schrieb:
> And yet, that "incorrect" behavior was clearly intended by the author(s) 
> of the code, test, and docstrings.
> 
> As it happens, Guido wrote that code (16 years ago) and the docstring (9 
> years ago), in the case of the posixpath module at least.

I don't find it that clear that it was the intention, AFAICT, it could 
have been an accident also. Guido added the doc strings as a 
contribution from Charles G. Waldman; he may just have documented the
implemented behavior.

In r4493, Sjoerd Mullender changed splitext (in an incompatible way)
so that it would split off only the last extension, before, foo.tar.gz
would be split into 'foo', '.tar.gz'. So it's clear that the intention
was always to "split off the extension", whether or not the behavior
on dotfiles was considered I cannot tell.

As for Doc/lib, in r6524 Guido changed it to document the actual 
behavior, from

the last component of \var{root} contains no periods,
and \var{ext} is empty or begins with a period.

to

and \var{ext} is empty or begins with a period and contains
at most one period.

So it seems the original (Guido's) intention was that it splits
of all extensions; Sjoerd then changed it to split off only the
last extension.

> And in an amusing twist, it appears that you yourself checked in the 
> test, 4 years ago!  If this behavior was so *obviously* buggy, why 
> didn't you ask Guido or "fix" it then?

If you look at #536661, you'll see that I committed that as a 
contribution of Sebastian Keim. The intention here was to rigorously
test the implemented behavior, after a failed attempt of challenging
it.

> But wait, it gets better!  Five years ago, you also recommended 
> *rejection* of a similar patch:
> 
> """I also dislike this patch. The current behaviour completely
> matches the documented behaviour; changing it might break
> existing applications. If you need a different behaviour,
> write a different function."""
> 
> http://python.org/sf/536120
> 
> So, why is it obviously broken now, but not five years ago?

I apparently took the same position that you now take back then,
whereas I'm now leaning towards (or going beyond) the position
Tim had back then, who wrote "BTW, if it *weren't* for the code 
breakage, I'd be in favor of doing this."

I now believe that this should be done *despite* having been
documented and tested (which, as you can see, was documented
and tested only match the implemented behavior). That it keeps
popping up is proof that the old behavior is deemed incorrect
by many people.

>> It's more than that. I conducted a poll, and here people were largely
>> in favor of that change. Had they been largely in opposition, I would
>> have rejected the patch.
> 
> By this logic, I could conduct a popularity poll for say, "fixing" the 
> distutils by changing its behavior to match that of setuptools, and go 
> ahead with it if the majority agreed with me that the distutils' 
> behavior was clearly incorrect by retroactive comparison -- despite it 
> being documented and tested behavior, and despite objections of backward 
> incompatibility being presented on Python-Dev.

And indeed, when you go back to the last discussion on distutils, this
is precisely what I recommend you to do, instead of putting layers
into setuptools to work around what you consider quirks in distutils.

> So reject it, or propose to add a new API.

Neither is a solution. Rejecting it means it will keep popping up
forever. The amount of Python code yet to be written is hopefully larger
than the code already written (paraphrasing Guido), so in the long run,
it should show the right behavior, not the historical one.

Adding a new API has a very high cost: it should ultimately involve
removing the old API, so *all* usage of splitext will eventually break,
not just those cases which happen to break under the current change.
Also, books need to be updated, etc. Furthermore, nobody has proposed
a name for a new API, yet.

>> P.S. If you apply the same effort to all changes that are constantly
>> being made to Python, you will find that you will need to revert many
>> of them.
> 
> Then I'm amazed that there is so much desire to *increase* the number of 
> changes being made to Python, if we can't even manage to follow our 
> policies *now*.

It seems that we disagree on what the policies are. 100% backwards
compatibility is none of them. Feature releases may include incompatible
changes, and this is just one of them (and a minor one, too). What
policy do you think is this change violating?

Regards,
Martin