Returning Windows file attribute information via os.stat()
Hi folks, As pointed out to me recently in an issue report [1] on my scandir module, Python's os.stat() simply discards most of the file attribute information fetched via the Win32 system calls. On Windows, os.stat() calls CreateFile to open the file and get the dwFileAttributes value, but it throws it all away except the FILE_ATTRIBUTE_DIRECTORY and FILE_ATTRIBUTE_READONLY bits. See CPython source at [2]. Given that os.stat() returns extended, platform-specific file attributes on Linux and OS X platforms (see [3] -- for example, st_blocks, st_rsize, etc), it seems that Windows is something of a second-class citizen here. There are several questions on StackOverflow about how to get this information on Windows, and one has to resort to ctypes. For example, [4]. To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows? Then, similarly to existing code like hasattr(st, 'st_blocks') on Linux, you could write a cross-platform function to determine if a file was hidden, something like so: FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h def is_hidden(path): if startswith(os.path.basename(path), '.'): return True st = os.stat(path) if hasattr(st, 'st_winattrs') and st.st_winattrs & FILE_ATTRIBUTE_HIDDEN: return True return False I'd be interested to hear people's thoughts on this. Thanks, Ben. [1]: https://github.com/benhoyt/scandir/issues/22 [2]: https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1462 [3]: https://docs.python.org/3.4/library/os.html#os.stat [4]: http://stackoverflow.com/a/6365265
On 10 June 2014 05:02, Ben Hoyt
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows?
+1. Given the precedent of Linux- and OS X-specific attributes, this seems like a no-brainer to me. Paul
2014-06-10 6:02 GMT+02:00 Ben Hoyt
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows? (...) FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h
if hasattr(st, 'st_winattrs') and st.st_winattrs & FILE_ATTRIBUTE_HIDDEN:
I don't like such API, it requires to import constants, use masks, etc. I would prefer something like: if st.win_hidden: ... Or maybe: if st.winattrs.hidden: ... Victor
FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h
if hasattr(st, 'st_winattrs') and st.st_winattrs & FILE_ATTRIBUTE_HIDDEN:
I don't like such API, it requires to import constants, use masks, etc.
I would prefer something like:
if st.win_hidden: ...
Or maybe:
if st.winattrs.hidden: ...
Yes, fair call. However, it looks like the precent for the attributes in os.stat()'s return value has long since been set -- this is OS-specific stuff. For example, what's in "st_flags"? It's not documented, but comes straight from the OS. Same with st_rdev, st_type, etc -- the documentation doesn't define them, and it looks like they're OS-specific values. I don't think the st.win_hidden approach gains us much, because the next person is going to ask for the FILE_ATTRIBUTE_ENCRYPTED or FILE_ATTRIBUTE_COMPRESSED flag. So we really need all the bits or nothing. I don't mind the st.st_winattrs.hidden approach, except that we'd need 17 sub-attributes, and they'd all have to be documented. And if Windows added another attribute, Python wouldn't have it, etc. So I think the OS-defined constant is the way to go. Because these are fixed-forever constants, I suspect in library code and the like people would just KISS and use an integer literal and a comment, avoiding the import/constant thing: if getattr(st, 'st_winattrs', 0) & 2: # FILE_ATTRIBUTE_HIDDEN ... -Ben
On 10 June 2014 13:19, Ben Hoyt
Because these are fixed-forever constants, I suspect in library code and the like people would just KISS and use an integer literal and a comment, avoiding the import/constant thing:
The stat module exposes a load of constants - why not add the (currently known) ones there? Finding the values of Windows constants if you don't have access to the C headers can be a pain, so having them defined *somewhere* as named values is useful. Paul
The stat module exposes a load of constants - why not add the (currently known) ones there? Finding the values of Windows constants if you don't have access to the C headers can be a pain, so having them defined *somewhere* as named values is useful.
So stat.FILE_ATTRIBUTES_HIDDEN and the like? Alternatively they could go in ctypes.wintypes, but I think stat makes more sense in this case. -Ben
On 2014-06-10 05:02, Ben Hoyt wrote: [snip]
FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h
def is_hidden(path): if startswith(os.path.basename(path), '.'): return True st = os.stat(path) if hasattr(st, 'st_winattrs') and st.st_winattrs & FILE_ATTRIBUTE_HIDDEN:
That could be written more succinctly as: if getattr(st, 'st_winattrs', 0) & FILE_ATTRIBUTE_HIDDEN:
return True return False
if hasattr(st, 'st_winattrs') and st.st_winattrs & FILE_ATTRIBUTE_HIDDEN:
That could be written more succinctly as:
if getattr(st, 'st_winattrs', 0) & FILE_ATTRIBUTE_HIDDEN:
return True return False
Yes, good call. Or one further: return getattr(st, 'st_winattrs', 0) & FILE_ATTRIBUTE_HIDDEN != 0 -Ben
On 06/09/2014 09:02 PM, Ben Hoyt wrote:
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows?
+1 to the idea, whatever the exact implementation. -- ~Ethan~
On Tue, Jun 10, 2014 at 12:17 PM, Ethan Furman
On 06/09/2014 09:02 PM, Ben Hoyt wrote:
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows?
+1 to the idea, whatever the exact implementation.
Agreed. -- Zach
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows?
+1 to the idea, whatever the exact implementation.
Cool. I think we should add a st_winattrs integer attribute (on Windows) and then also add the FILE_ATTRIBUTES_* constants to stat.py per Paul Moore. What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests? -Ben
On Tue, Jun 10, 2014 at 2:04 PM, Ben Hoyt
To solve this problem, what do people think about adding an "st_winattrs" attribute to the object returned by os.stat() on Windows?
+1 to the idea, whatever the exact implementation.
Cool.
I think we should add a st_winattrs integer attribute (on Windows) and then also add the FILE_ATTRIBUTES_* constants to stat.py per Paul Moore.
Add to _stat.c rather than stat.py.
What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests?
Yep! -- Zach
What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests?
Yep!
Okay, I've done step one (opened an issue on bugs.python.org), and hope to provide a patch in the next few weeks if no-one else does (I've never compiled CPython on Windows before): http://bugs.python.org/issue21719 -Ben
On 6/11/2014 9:27 AM, Ben Hoyt wrote:
What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests?
Yep!
Okay, I've done step one (opened an issue on bugs.python.org), and hope to provide a patch in the next few weeks if no-one else does (I've never compiled CPython on Windows before):
If you have problems compiling, the core-mentorship list is one place to ask. For 3.4+, I believe the devguide instructions are correct. If not, say something. -- Terry Jan Reedy
participants (7)
-
Ben Hoyt -
Ethan Furman -
MRAB -
Paul Moore -
Terry Reedy -
Victor Stinner -
Zachary Ware