[New-bugs-announce] [issue39312] Expose placeholder reparse points in Windows

[Eryk Sun]

New submission from Eryk Sun <eryksun at gmail.com>:

Windows 10 apparently defaults to disguising placeholder reparse points in python.exe processes, but exposes them to cmd.exe and powershell.exe processes. 

A common example is a user's OneDrive folder, which extensively uses placeholder reparse points for files and directories. The placeholder file attributes include FILE_ATTRIBUTE_REPARSE_POINT, FILE_ATTRIBUTE_OFFLINE, and FILE_ATTRIBUTE_SPARSE_FILE, and the reparse tags are in the set
IO_REPARSE_TAG_CLOUD[_1-F] (0x9000[0-F]01A). Currently, we don't see any of this information in a python.exe process when we call FindFirstFile[Ex]W, GetFileAttributesW, or query file information on a file opened with FILE_FLAG_OPEN_REPARSE_POINT, such as when we call os.lstat. 

The behavior is determined by the process or per-thread placeholder-compatibility mode. The process mode can be queried via RtlQueryProcessPlaceholderCompatibilityMode [1]. The documentation says that "[m]ost Windows applications see exposed placeholders by default". I don't know what criteria Windows is using here, but in my tests with python.exe and a simple command-line test program, the default mode is PHCM_DISGUISE_PLACEHOLDER.

Should Python provide some way to call RtlSetProcessPlaceholderCompatibilityMode [2] to set PHCM_EXPOSE_PLACEHOLDERS mode for the current process? Should os.lstat be modified to temporarily expose placeholders -- for the current thread only -- via RtlSetThreadPlaceholderCompatibilityMode [3]? We can dynamically link to this ntdll function via GetProcAddress. It returns the previous mode, which we can restore after querying the file.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlqueryprocessplaceholdercompatibilitymode
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlsetprocessplaceholdercompatibilitymode
[3] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlsetthreadplaceholdercompatibilitymode

----------
components: Library (Lib), Windows
messages: 359850
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Expose placeholder reparse points in Windows
type: enhancement
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39312>
_______________________________________