On Fri, 06 Sep 2013 06:50:07 +0100 Chris Withers wrote:

So, from my perspective, I'm either looking at a bug in shutil.rmtree (which would be trying to delete a directory before deleting its content or failing to delete a file but ignoring an error) or the file object, when being used as a context manager, going through __exit__ without closing the file and releasing the handle.

It seems someone forgot to remove the following snippet from FileIO.__exit__: if random.random() > 0.0001: # Sometimes we leave the fd open, to annoy Chris os.close(self.fd)

This happens very infrequently, the OS is Windows 7 and the filesystem is NTFS, if that helps...

It should help indeed: http://blogs.msdn.com/b/oldnewthing/archive/2012/09/07/10347136.aspx :-) Regards Antoine.

Antoine Pitrou writes:

http://blogs.msdn.com/b/oldnewthing/archive/2012/09/07/10347136.aspx

That was worth it just for the comment from Australia!

On 06/09/2013 08:14, Nick Coghlan wrote:

On 6 September 2013 15:50, Chris Withers wrote:

...

Continuous testing is a wonderful thing when it comes to finding weird edge case problems, like this one:

[... snip ...]

...

os.rmdir(path) OSError: [WinError 145] The directory is not empty: 'c:\\users\\jenkins\\appdata\\local\\temp\\tmpkeg4d7\\a'

This feels a lot like an issue we were seeing on the Windows buildbots, which we ended up working around in the test support library: http://bugs.python.org/issue15496

That would be some awfully ugly code to upgrade from "hack in the test support library" to "this is just how Python unlinks files on Windows", though :P

I think that any kind of delay-retry loop falls squarely within the remit of the calling application, not core Python. This isn't a problem of Python's making: IIUC, you would see the same effect if you used any other language or simply deleted the folder from within Explorer. (I don't know whether Explorer itself does anything canny under the covers to retry). Obviously our test suite *is* a calling application, and so it makes perfect sense to put some workaround in place. The trouble with this class of problem, where a share-delete handle allows operations to succeed and to fail later which would normally fail early, is that the bad effect is at one remove from its cause. Here, by the time the rmtree has reached the point of removing a parent directory, it's long-since left behind the file which has a still-open handle: the DeleteFile succeeded and the code moved on. You can't even tell which file it was. A related problem arises where the DeleteFile succeeds and an error occurs when a subsequent CreateFile fails for the same filepath, again because a share-delete handle is still open for a file at that path. This is another one which hits our test suite because of an overuse of one temp filename. What should Python do? With some effort it could look for open file handles against the file it's trying to delete, but what then? Wait until they're all closed? That could leave it hanging. And even with a timeout it would introduce a delay which might be unnecessary. A lot of the time, no harm will come of the file existing a few seconds after the DeleteFile has succeeded. In short, I don't believe there's any mileage in introducing extra code into Python's os or io modules. It falls on the shoulders of the calling code to implement retry loops or whatever logic as needed. TJG

Le Fri, 06 Sep 2013 08:58:06 +0100, Tim Golden a écrit :

What should Python do?

Maybe using FILE_SHARE_DELETE could help? http://bugs.python.org/issue15244 Regards Antoine.

On 06/09/2013 11:23am, Tim Golden wrote:

On 06/09/2013 11:14, Antoine Pitrou wrote:

...

Le Fri, 06 Sep 2013 08:58:06 +0100, Tim Golden a écrit :

...

What should Python do?

Maybe using FILE_SHARE_DELETE could help? http://bugs.python.org/issue15244

I don't think so. It's the use of FILE_SHARE_DELETE (by other programs, eg Virus Checkers) that typically causes the problem. IOW, the sequence is:

* [Some Other Prog] takes FILE_SHARE_DELETE handle, allowing other programs to delete the file even while this handle is still open

* Python calls DeleteFile -- succeeds if the only open handles are FILE_SHARE_DELETE; carries on

* File has apparently disappeared but still has open handles

* Any attempt to create a file of the same name or to delete a containing directory fail because the file is still open, even though it's successfully been deleted.

* (Some time later) [Some Other Prog] closes its handle and the file is now completely gone

Unless I'm missing something, there's nothing Python can do to help here apart from the sort of delay-retry dance which test.support uses and which I'm advocating against as a core feature.

Instead of deleting, the file could be moved to a temporary name in the root directory (or some other permanent directory on the same drive) and then deleted. That would allow the directory to be closed even if a FILE_SHARE_DELETE handle is still open for the file. -- Richard

On 6 September 2013 13:21, Tim Golden wrote:

True, but then you're into determining a temporary name somewhere on the same volume if possible and avoiding collisions etc. Again, I don't think this is something we need to be doing by default in core Python.

Agreed. There simply is no solution that works in all cases. If you rename the file to anywhere but the same directory, you potentially have permission issues. And if you rename to the same directory, you still can't delete the directory. It's a shame, because a helper to do this would be useful for Unix users wanting to ensure that their code works properly on Windows, but there really isn't an answer other than knowing how things work (and crafting an appropriate solution for your application), here... Paul

On Fri, 06 Sep 2013 13:21:35 +0100, Tim Golden wrote:

On 06/09/2013 12:50, Richard Oudkerk wrote:

...

On 06/09/2013 11:23am, Tim Golden wrote:

...

On 06/09/2013 11:14, Antoine Pitrou wrote:

...

Le Fri, 06 Sep 2013 08:58:06 +0100, Tim Golden a écrit :

...

What should Python do?

Maybe using FILE_SHARE_DELETE could help? http://bugs.python.org/issue15244

I don't think so. It's the use of FILE_SHARE_DELETE (by other programs, eg Virus Checkers) that typically causes the problem. IOW, the sequence is:

* [Some Other Prog] takes FILE_SHARE_DELETE handle, allowing other programs to delete the file even while this handle is still open

* Python calls DeleteFile -- succeeds if the only open handles are FILE_SHARE_DELETE; carries on

* File has apparently disappeared but still has open handles

* Any attempt to create a file of the same name or to delete a containing directory fail because the file is still open, even though it's successfully been deleted.

* (Some time later) [Some Other Prog] closes its handle and the file is now completely gone

Unless I'm missing something, there's nothing Python can do to help here apart from the sort of delay-retry dance which test.support uses and which I'm advocating against as a core feature.

Instead of deleting, the file could be moved to a temporary name in the root directory (or some other permanent directory on the same drive) and then deleted. That would allow the directory to be closed even if a FILE_SHARE_DELETE handle is still open for the file.

True, but then you're into determining a temporary name somewhere on the same volume if possible and avoiding collisions etc. Again, I don't think this is something we need to be doing by default in core Python.

What about moving the test.support delete helper to somewhere in unittest, since this will come up in pretty much every test suite that runs on Windows? --David