[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

[Ronald Oussoren]

On 10 May, 2013, at 15:54, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Fri, 10 May 2013 15:46:21 +0200,
> Christian Heimes <christian at python.org> a écrit :
> 
>> Am 10.05.2013 14:16, schrieb Antoine Pitrou:
>>> But what if some systems return more than the file type and less
>>> than a full stat result? The general problem is POSIX's terrible
>>> inertia. I feel that a stat result with some None fields would be
>>> an acceptable compromise here.
>> 
>> POSIX only defines the d_ino and d_name members of struct dirent.
>> Linux, BSD and probably some other platforms also happen to provide
>> d_type. The other members of struct dirent (d_reclen, d_namlen)
>> aren't useful in Python space by themselves.
>> 
>> d_type and st_mode aren't compatible in any way. As you know st_mode
>> also contains POSIX permission information. The file type is encoded
>> with a different set of bits, too. Future file types aren't mapped to
>> S_IF* constants for st_mode.
> 
> Thank you and Ronald for clarifying. This does make the API design a
> bit bothersome. We want to expose as much information as possible in a
> cross-platform way and with a flexible granularity, but doing so might
> require a gazillion of namedtuple fields (platonically, as much as one
> field per stat bit).

One field per stat bit is overkill, file permissions are well known enough
to keep them as a single item. 

Most if not all uses of the st_mode field can be covered by adding just
"filetype" and "permissions" fields. That would also make it possible to
use stat_result in os.scandir() without loosing information (it would 
have filetype != None and permissions and st_mode == None on systems with
d_type).

> 
>> For d_ino you also need the device number from the directory because
>> the inode is only unique within a device.
> 
> But hopefully you've already stat'ed the directory ;)

Why? There's no need to stat the directory when implementing os.walk using
os.scandir (for systems that return filetype information in the API used
by os.scandir).  Anyway, setting st_ino in the result of os.scandir is
harmless, even though using st_ino is uncommon.

Getting st_dev from the directory isn't good anyway, for example when
using rebind mounts to mount a single file into a different directory (which 
is a convenient way to make a configuration file available in a chroot environment)

Ronald

> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com