[Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()

[Brett Cannon]

On Tue, 19 Apr 2016 at 04:46 Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Brett Cannon writes:
>  > On Mon, 18 Apr 2016 at 12:26 Stephen J. Turnbull <stephen at xemacs.org>
> wrote:
>
>  > Well, it makes *your* head hurt;
>
> It doesn't, because I have a different (and IMHO better) model.  I can
> interpret yours without pain by comparing to that.
>
>  > By providing os.fspath() I can say that I do not, under any
>  > circumstances, want someone to guess at the encoding some bytes
>  > path is under to get me a string and instead I want to start and
>  > end entirely in a world of strings. IOW os.fspath() lets me work in
>  > such a way that the instant bytes are introduced into my code for
>  > file paths it triggers a TypeError.
>
> Does it really help you work that way?  open is polymorphic, and will
> use os._raw_fspath(obj, (bytes,str)).  Ditto os.scandir etc.  If they
> don't, there's no point in supporting bytes returns from __fspath__,
> is there?


You're leaving out all of the os.path functions, but you're right that if
they didn't support it like Windows then this entire discussion of bytes
paths would be moot.


>   Application code will normally not be calling os.fspath.
> In the future, pathlib will, I suppose, but even without os.fspath
> pathlib already protects you, as does antipathy.[1]
>

I disagree that application code won't be calling os.fspath.


>
> More effective, then, is just to use pathlib for your Path-hacking
> work as soon as the path-representing object appears, and Path will
> complain about bytes for you.  This is an analogue of the "decode
> bytes at the boundary" principle.
>

Ah, but you see that doesn't make porting easy. If I have a bunch of
path-manipulating code using os.path already and I want to add support for
pathlib I can either (a) rewrite all of that path-manipulating code to work
using pathlib, or (b) simply call `path = os.fspath(path)` and be done with
it. Basically if you have written any code that uses os.path then you will
have to care about (a) or (b) as a way to add support for pathlib short of
the `str(path)` hack we're all working to get away from. And if people
truly liked option (a) then this conversation wouldn't be such a big deal
as we would have seen more people using pathlib already (yes, the
provisional tag may have scared some off, but my guess is it's more from
not wanting to rewrite os.path-using code).

Now if you can convince me that the use of bytes paths is very minimal and
thus people doing path manipulations with them will be a very small
minority then I'm happy to try and use this to keep pushing people towards
avoiding bytes for file paths. But over the years people such as yourself,
Stephen, have convinced me that people do some really crazy stuff with
their file systems and that it isn't isolated to just one or two people.
And so it becomes this situation where we need to ask ourselves if we are
going to tell them to just deal with it or help them transition.

The other way to convince me is that people needing to support older
versions of Python will use `path = path.__fspath__() if hasattr(path,
'__fspath__') else path` and that allowing bytes with that idiom is going
to cost them dearly. My current assumption is that it won't because people
using that idiom are using os.path and those functions will complain when
mixing str and bytes together, but I'm open to being convinced otherwise.

I guess what I'm trying to get at is that I understand the desire to get
people to get the bytes path habit, but to me the best way will be to get
people quickly and easily transitioned over to pathlib as a carrot rather
than using the lack of bytes path support in this transition as a stick.

-Brett



>
>  > Yep, we are stuck with the names unless you want to propose a new
>  > name and deprecate the old one.
>
> I already proposed fs_ensure_bytes and fs_ensure_str.  I think they're
> sufficiently ugly to prove my point.<wink/>
>
>
> Footnotes:
> [1]  Strictly speaking, antipathy protects you from inadvertant mixing
> of bytes and str.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160419/531d2be8/attachment.html>