[Python-ideas] tweaking the file system path protocol

[Wolfgang Maier]

On 05/23/2017 06:17 PM, Koos Zevenhoven wrote:
> On Tue, May 23, 2017 at 1:12 PM, Wolfgang Maier
> <wolfgang.maier at biologie.uni-freiburg.de> wrote:
>> What do you think of this idea for a slight modification to os.fspath:
>> the current version checks whether its arg is an instance of str, bytes or
>> any subclass and, if so, returns the arg unchanged. In all other cases it
>> tries to call the type's __fspath__ method to see if it can get str, bytes,
>> or a subclass thereof this way.
>>
>> My proposal is to change this to:
>> 1) check whether the type of the argument is str or bytes *exactly*; if so,
>> return the argument unchanged
>> 2) check wether __fspath__ can be called on the type and returns an instance
>> of str, bytes, or any subclass (just like in the current version)
>> 3) check whether the type is a subclass of str or bytes and, if so, return
>> it unchanged
> 

Hi Koos and thanks for your detailed response,

> The reason why this was not done was that a str or bytes subclass that
> implements __fspath__(self) would work in both pre-3.6 and 3.6+ but
> behave differently. This would be also be incompatible with existing
> code using str(path) for compatibility with the stdlib (the old way,
> which people still use for pre-3.6 compatibility even in new code).
> 

I'm not sure that sounds very convincing because that exact problem 
exists, was discussed and accepted in your PEP 519 for all other 
classes. I do not really see why subclasses of str and bytes should 
require special backwards compatibility here. Is there a reason why you 
are thinking they should be treated specially?

>> This would have the following implications:
>> a) it would speed up the very common case when the arg is either a str or a
>> bytes instance exactly
> 
> To get the same performance benefit for str and bytes, but without
> changing functionality, there could first be the exact type check and
> then the isinstance check. This would add some performance penalty for
> PathLike objects. Removing the isinstance part of the __fspath__()
> return value, which I find less useful, would compensate for that. (3)
> would not be necessary in this version.
> 

Right, that was one thing I forgot to mention in my list. My proposal 
would also speed up processing of pathlike objects because it moves the 
__fspath__ call up in front of the isinstance check. Your alternative 
would speed up only str and bytes, but would slow down Path-like classes.
In addition, I'm not sure that removing the isinstance check on the 
return value of __fspath__() is a good idea because that would mean 
giving up the guarantee that os.fspath returns an instance of str or 
bytes and would effectively force library code to do the isinstance 
check anyway even if the function may have performed it already, which 
would worsen performance further.

> Are you asking for other reasons, or because you actually have a use
> case where this matters? If this performance really matters somewhere,
> the version I describe above could be considered. It would have 100%
> backwards compatibility, or a little less (99% ?) if the isinstance
> check of the __fspath__() return value is removed for performance
> compensation.
> 

That use case question is somewhat difficult to answer. I had this idea 
when working on two bug tracker issues (one concerning fnmatch and a 
follow-up one on os.path.normcase, which is called by fnmatch.filter 
and, in turn, calls os.fspath. fnmatchfilter is a case where performance 
matters and the decision when and where to call the rather expensive 
os.path.normcase->os.fspath there is not entirely straightforward. So, 
yes, I was basically looking at this because of a potential use case, 
but I say potential because I'm far from sure that any speed gain in 
os.fspath will be big enough to be useful for fnmatch.filter in the end.


>> b) user-defined classes that inherit from str or bytes could control their
>> path representation just like any other class
> 
> Again, this would cause differences in behavior between different
> Python versions, and based on whether str(path) is used or not.
> 
> —Koos
> 
>