[Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue

[Guido van Rossum]

On Tue, Sep 30, 2008 at 7:53 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 30 Sep 2008 11:50:10 pm Guido van Rossum wrote:
>
>> > To avoid silent skipping, is it possible to drop 'unreadable'
>> > names, issue a warning (instead of exception), and continue to
>> > completion? "Warning: unreadable filename skipped; see
>> > PyWiki/UnreadableFilenames"
>>
>> That would be annoying as hell in most cases.
>
> Doesn't the warning module default to only displaying the warning once
> per session? I don't see that it would be annoying as hell to be
> notified once per session that an error has occurred.
>
> What I'd find annoying as hell would be something like this:
>
> $ ls . | wc -l
> 25
> $ python
> ...
>>>> import os
>>>> len(os.listdir('.')
> 24

And yet similar discrepancies happen all the time -- ls suppresses
filenames starting with '.', while os.listdir() shows them (except '.'
and '..' themselves). The Mac Finder and its Windows equivalent hide
lots of files from you. And have you considered mount points (on
Unix)?

Face it. Filesystems are black boxes. They have roughly specified
behavior, but calls into them can fail or seem inconsistent for many
reasons -- concurrent changes by other processes, hidden files
(Windows), files that exist but can't be opened due to kernel-level
locking, etc. It's best not to worry too much about this.

Here's another anomaly:

>>> import os
>>> '.snapshot' in os.listdir('.')
False
>>> os.chdir('.snapshot')
>>> os.getcwd()
'/home/guido/bin/.snapshot'
>>>

IOW there's a hidden .snapshot directory that os.listdir() doesn't
return -- but it exists! This is a standard feature on NetApp filers.
(The reason this file is extra hidden is that it gives access to an
infinite set of backups that you don't want to be found by find(1),
os.walk() and their kin.)

> Give me a nice clear error, or even a warning. Don't let the error pass
> silently, unless I explicitly silence it.

Depends on your use case. We're talking here of a family of APIs where
different programs have different needs. I assert that most programs
are best served by an API that doesn't give them surprising and
irrelevant errors, as long as there's also an API for the few that
want to get to the bottom of things (or as close as they can get --
see above '.snapshot' example).

>> I consider the dropping of unreadable names similar to the
>> suppression of "hidden" files by various operating systems.
>
> With the exception of '.' and '..', I consider "hidden" files to be a
> serious design mistake, but at least most operating systems give the
> user a way to easily see all such hidden files if you ask.
>
> (Almost all. Windows has "superhidden" files that remain hidden even
> when the user asks to see hidden files, all the better to hide malware.
> But that's a rant for another list.)

Rant all you want, it won't go away.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)