[Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()

[Stephen J. Turnbull]

Nick Coghlan writes:

 > On 15 April 2016 at 00:52, Stephen J. Turnbull <stephen at xemacs.org> wrote:
 > > Nick Coghlan writes:
 > >
 > >  > The use case for returning bytes from __fspath__ is DirEntry, so you
 > >  > can write things like this in low level code:
 > >  >
 > >  >     def myscandir(dirpath):
 > >  >         for entry in os.scandir(dirpath):
 > >  >             if entry.is_file():
 > >  >                 with open(entry) as f:
 > >  >                     # do something
 > >
 > > Excuse me, but that is *not* a use case for returning bytes from
 > > DirEntry.__fspath__.  open() is perfectly happy taking str (including
 > > surrogate-encoded rawbytes).
 > 
 > That results in a different type for the file object's name:
 > 
 > >>> open("README.md").name
 > 'README.md'
 > >>> open(b"README.md").name
 > b'README.md'

OK, you win, __fspath__ needs to be polymorphic.

But you've just shifted me to -1 on "os.fspath": it's an attractive
nuisance.  EIBTI, applications and high-level library functions should
use os.fsdecode or os.fsencode.  Functions that take a polymorphic
argument and want preserve type should invoke __fspath__ on the
argument.  That will visually signal that the caller is not merely
low-level, but is explicitly a boundary function.  (You could rename
the generic function as "os._fspath", I guess, but I *really* want to
deprecate calling the polymorphic version in user code.  _fspath can
be added if experience shows that polymorphic usage is very desireable
outside the stdlib.  This remark is in my not-so-Dutch opinion, of
course.)

 > The guarantee we want to provide those folks is that if they're
 > operating in the binary domain they'll stay there.

Et tu, Nick?  "Guarantee"?!  You can't guarantee any such thing with
an implicitly invoked polymorphic API like this one -- unless you
consider a crashed program to be in the binary domain. ;-)  Note that
the current proposala don't even do that for the binary domain, only
for the text domain!