[Python-ideas] tweaking the file system path protocol

Nick Coghlan ncoghlan at gmail.com
Sun May 28 02:15:32 EDT 2017

On 28 May 2017 at 15:18, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, May 26, 2017 at 03:58:23PM +0300, Koos Zevenhoven wrote:
> I think this argument about backwards compatibility is a storm in a tea
> cup. We can enumerate all the possibilities:
> 1. object that doesn't inherit from str/bytes: behaviour is unchanged;
> 2. object that does inherit from str/bytes, but doesn't override
>    the __fspath__ method: behaviour is unchanged;
> 3. object that inherits from str/bytes, *and* overrides the __fspath__
>    method: behaviour is changed.
> Okay, the behaviour changes. I doubt that there will be many
> classes that subclass str and override __fspath__ now, because
> that would have been a waste of time up to now. So the main risk is:
> - classes created from Python 3.7 onwards;
> - which inherit from str/bytes;
> - and which override __fspath__;
> - and are back-ported to 3.6;
> - without taking into account that __fspath__ will be ignored in 3.6;
> - and the users don't read the docs to learn about the difference.
> The danger here is the possibility that the wrong pathname will be used,
> if str(obj) and fspath(obj) return a different string.
> Personally I think this is unlikely and not worth worrying about beyond a note in
> the documentation, but if people really feel this is a problem we could
> make this a __future__ import. But that just feels like overkill.

It wouldn't even need to be a __future__ import, as we have a runtime
warning category specifically for this kind of change:

So *if* a change like this was made, the appropriate transition plan would be:

Python 3.7: at *class definition time*, we emit FutureWarning for
subclasses of str and bytes that define __fspath__, saying that it is
currently ignored for such subclasses, but will be called in Python
Python 3.8: os.fspath() is changed as Wolgang proposes, such that
explicit protocol support takes precedence over builtin inheritance

However, if we *did* make such a change, it should also be made for
operator.index as well, since that is similarly inconsistent with the
way the int/float/etc constructor protocols work:

    >>> from operator import index
    >>> class MyInt(int):
    ...     def __int__(self):
    ...         return 5
    ...     def __index__(self):
    ...         return 5
    >>> int(MyInt(10))
    >>> index(MyInt(10))
    >>> class MyFloat(float):
    ...     def __float__(self):
    ...         return 5.0
    >>> float(MyFloat(10))
    >>> class MyComplex(complex):
    ...     def __complex__(self):
    ...         return 5j
    >>> complex(MyComplex(10j))
    >>> class MyStr(str):
    ...     def __str__(self):
    ...         return "Hello"
    >>> str(MyStr("Not hello"))
    >>> class MyBytes(bytes):
    ...     def __bytes__(self):
    ...         return b"Hello"
    >>> bytes(MyBytes(b"Not hello"))


P.S. I'll also echo Steven's observations that it is entirely
inappropriate to describe the thinking of other posters to the list as
being overly shallow. The entire reason we *have* python-ideas and the
PEP process is because programming language design is a *hard
problem*, especially for a language with as broad a set of use cases
as Python. Rather than trying to somehow survey the entire world of
Python developers, we instead provide them with an open forum where
they can say "This surprises or otherwise causes problems for me" and
describe their perspective. That's neither deep nor shallow thinking,
it's just different people using the same language in different ways,
and hence encountering different pain points.

As far as the specific point at hand goes, I think contrasting the
behaviour of PEP 357 (__index__) and PEP 519 (__fspath__) with the
behaviour of the builtin constructor protocols suggest that this is
better characterised as an oversight in the design of the more recent
protocols, since neither PEP explicitly discusses the problem, both
PEPs were specifically designed to permit the use of objects that
*don't* inherit from the relevant builtin types (since subclasses
already worked), and both PEPs handle the "subclass that also
implements the corresponding protocol" scenario differently from the
way the builtin constructor protocols handle it.

Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list