ok, i can actually chime in here because i've done filesystems work on windows (don't ask ;). now, it's been a while, but i should remember things reasonably accurately (i hope). see below for comments: ----- Original Message ----- From: Moof To: twisted-python@twistedmatrix.com Sent: Saturday, December 31, 2005 2:15 AM Subject: Re: [Twisted-Python] Twisted windows hackers - help the tests to pass!
It's not so much a misconfiguration, as an issue with the fact that trial on Window s is failing to rename its _trial_temp folder, because there are files inside it that are open. I've opened a bug on it over on < http://twistedmatrix.com/bugs/issue1387>.
it is incorrect to state that file deletion will always fail when the file is open. let me explain how nt/ntfs/win32 does this, because it's highly 'strange' to folks from a posix background. this may be way more info than you were looking for, but i'm going to type it out anyway so that someone can then make the call on how to deal with the issue. for functionality implemented in the kernel, such as ntfs, windows has 2 api layers: win32 (you see CreateFile() here) and the native nt api. win32 is implemented in usermode, while the native api is implemented as a kernel service, which is exposed to userspace code with the Zw prefix (ie ZwCreateFile()). sometimes the win32 api calls map directly, sometimes they are multiplexed (ie one win32 api call may use multiple native api calls to do the work). what happens here is undocumented (at least officially and without an nda). what is important is that some calls allow you to do things which win32 does not expose; most popular ones deal with io completion (this is how you cancel async i/o on windows nt). now, on to the meat: the win32 CreateFile() is actually a jack-of-all-trades call (horrible design), used for file creation, opening *and authorization*. we're not interested in ACLs in this case, but we are interested in locks - locks in nt are compulsory (as opposed to purely advisory in posix) and violating one will result in an auth failure. now, disregarding the ability to take out range locks (locks on a byterange within a file), CreateFile() takes out certain locks based on the sharemode (iirc, not sure) parameter. the rules here are funky, iirc, but the one we care about is FILE_SHARE_DELETE. *unless* this flag is set in sharemode passed to CreateFile(), all attempts to open this file for deletion (see, authorization) going forward will fail. it will fail when the file is opened exclusively. CreateFile takes a parameter called sharemode, iirc, which can be a combination of FILE_SHARE_READ, FILE_SHARE_WRITE, and FILE_SHARE_DELETE. *unless* FILE_SHARE_DELETE is set in the flag parameter, a lock preventing deletion gets taken out and you get to enjoy all the wonders of compulsory locking you are hitting. here's why i think this happens: win32 DeleteFile() does *not* actually use ZwDeleteFile() native call to delete a file. instead, it does a CreateFile() (or ZwCreateFile(), could be either) to open the file (and get a handle to it), telling it the desired access is DELETE. then it does a ZwSetFileInformation(), which it tells it wants to set 'disposition' and passes in the appropriate disposition info struct with the DELETE disposition set. you can't cheat this, because ZwSetInformation() will fail if the handle doesn't have DELETE rights, and you won't get them if the file isn't opened with FILE_SHARE_DELETE. bummer. the reason this is all so roundabout is that the file whose disposition is set to DELETE doesn't actually get deleted until the last handle is ZwClose()'d - this is where the deletion takes place. now, this is from memory, so don't hold me to it exactly, but the jist of it should be correct. the interesting, albeit non-obvious, question is: what does ZwDeleteFile() do? it takes either a handle *or a path* and deletes it *right away*, without waiting for handles to be closed. now, i don't know whether it bypasses the ZwCreateFile() and hence the DELETE check, but there's a chance that it is the call ZwClose() makes internally when it does a delete based on the disposition (this guess would be supported by the fact that ZwCreateFile() and a few other fs calls are documented in msdn/ddk, but ZwDeleteFile() is not) and hence doesn't hit this check. what is even more interesting is that, in win32, the only way to remove a directory is using RemoveDirectory(), which requires the directory to be empty. you have to recursively delete all contents, either by hand or using SHFileOperation, iirc. this is the op that hits your open files problem. i remember seeing code (this part i didn't work on, but did read brielfy) which deleted directories with ZwDeleteFile() and there was no resurive content deletion code, so i suspect that you could delete a non-empty directory this way. neither of this is valid ntfs usage, so doing it may not be kosher. whatever the case may be, those two options are available if you can stomach them.
It's going to need a rethink on how _trial_temp works, because my instant thought on how to solve was "symlinks" and python doesn't support them on windows, mostly because windows' own support of them is a tad on the "dont' ask, don't tell" side of things, and NTFS-only anyway.
NTFS and DFS, but yeah, no FAT (if anyone cares). symlinks in ntfs can be implemented using what's called 'reparse points'. these are actually quite powerful - you can attach either a static transformation or code (you need a driver for this, iirc) to a certain dentry these symlinks are called 'junctions', but they work only for directories. currently, all of this is extremely hairy to use. there are also hardlinks, which you can actually create with the win32 api, but they are only for files moreover, none of this behaves like symlinks and hardlinks in terms of finer semantics (ie unlinking). junctions are most closely related to mount --bind, rather than symlinks, for example. with all of the above said, you've basically got these choices (in no particular order) to deal with the issue at hand: 1. change the offending code not to do this rename 2. instead of a rename, create a new directory, do a recursive copy into it from the original and retry removing the original directory asynchronously until it succeeds. obviously, the file handles which are open at that point will be referencing a different copy of the files from the ones which will be opened subsequently. 3. test whether the zwDeleteFile() behaves in the way i conjrectured it to (wrt files or directories). if so, cause it to be implemented in the pywin32 extension and use it to perform the delete. 4. test whether you can either use ZwSetFileInformation() to rename directories by changing the FILE_NAME attr in the appropriate info structure or use it to move by renaming files which are open, again using the appropriate (but different) structure. if so, implement this or cause this to be implemented in pywin32. it is unclear (to me) whether this would result in the pre-move file handles being dead, stale or correct. 5. instead of opening the files as normal, open them with pywin32's implementation of CreateFile(), specifying the appropriate sharemode. this will allow the rename (move really) to go through, but it is unclear what happens to the preexisting filehandles. 6. implement, or cause to be implemented, CreateHardlink() in pywin32. create a new directory and recursively hardlink contents of the original into the new directory. asynchronously retry recursive deletion of the original one. that's all i can think of anyway.
MFen tracked down an error involving "r+b" mode. Seems windows handling of it is insane. See ><http://twistedmatrix.com/bugs/issue1386>. This could well be a python bug, or a feature of windows.
i'm not sure how python does file opens and i/o on windows. with that said, assuming that it uses fopen() from the visual studio c runtime library, there is a quirk in the implementation that might be causing this. if you use any of the + modes, ie a+, r+ or w+. when you switch between reading and writing you need to do an fflush() or fsetpos() (possibly some others like fseek() could work too, don't remember). try doing a file.flush() on your file object somewhere in there and see if that fixes things for you. if it does, this should probably be reported as a python stdlib bug. it's really late, so pardon the wordiness and possible inaccuracies due to memory lapses. hth, -p