[ python-Bugs-1425127 ] os.remove OSError: [Errno 13] Permission denied

[SourceForge.net]

Bugs item #1425127, was opened at 2006-02-06 10:44
Message generated for change (Comment added) made by atila-cheops
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1425127&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: cheops (atila-cheops)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.remove OSError: [Errno 13] Permission denied

Initial Comment:
When running the following program I get frequent
errors like this one

Exception in thread Thread-4:
Traceback (most recent call last):
  File "C:\Python24\lib\threading.py", line 442, in
__bootstrap
    self.run()
  File "os.remove.py", line 25, in run
    os.remove(filename)
OSError: [Errno 13] Permission denied:
'c:\\docume~1\\joag\\locals~1\\temp\\tmpx91tkx'

When leaving out the touch statement(line 24) in the
loop of the class, I do not get any errors.
This is on Windows XP SP2 with python-2.4.2 (you should
have an exe touch somewhere in you path for this to
work) Can somebody shed any light on this please?

Thanks in advance

Joram Agten



----------------------------------------------------------------------

>Comment By: cheops (atila-cheops)
Date: 2006-02-07 11:32

Message:
Logged In: YES 
user_id=1276121

for the subprocess.py I did the following in a few places

try:
    _active.remove(self)
except:
    pass

see also bug 1199282
https://sourceforge.net/tracker/index.php?func=detail&aid=1199282&group_id=5470&atid=105470

in my current script I circumvent the "Permission denied"
error in the following way:

removed = False
while not removed:
try:
    os.remove(file)
    except OSError, error:
        logger.warning("could not remove file %s, %s"
%(file, error))
        time.sleep(1)
    else:
        removed = True

I also have a virus scanner (Mcafee, corporate stuff), and
still get the same behaviour when disabling the virus scanner.

>My feel, after staring at filemon output, is that this is a
>problem in the Windows file I/O layer.  NTFS queues the
>various operations, and calling an external process with
>stuff still in the queue messes up the request scheduling.

this seems strange to me, since every thread works with its
own temp files, and all requests are send one after another
to the file I/O layer

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2006-02-07 07:59

Message:
Logged In: YES 
user_id=38376

"Does it tell you more than _just_ that?  It doesn't for me."

All requests against the file in question were issued by the
python process; there's no sign of virus checkers or other
external applications.

Also, whenever things failed, there were always multiple
requests for cmd.exe (caused by os.system) between the WRITE
request and the failing OPEN request.
 
My feel, after staring at filemon output, is that this is a
problem in the Windows file I/O layer.  NTFS queues the
various operations, and calling an external process with
stuff still in the queue messes up the request scheduling.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2006-02-07 04:07

Message:
Logged In: YES 
user_id=31435

[/F]
> Except that he's hitting the file system quite heavily,

Except that _without_ the call to touch(), he's hitting it
even more heavily, creating and destroying little files just
as fast as the OS can do it in each of 10 threads -- but
there aren't any errors then.

> and asyncronously.

What's asynch here?  The OP's touch() function waits for the
spawned process to terminate, and the test driver doesn't
try to delete the file until after that.

> My guess is that Windows simply gets behind
> (a quick filemon test indicates that this is indeed the
> case; just before a crash, I see the events
> CREATE/SUCCESS, QUERY/SUCCESS, QUERY/SUCCESS,
> WRITE/SUCCESS, and OPEN/SHARING VIOLATION for the
> failing file, with lots of requests for other files
> interleaved).

That's consistent with the symptom reported:  an exception
raised upon trying to remove the file, but not during any
other file operation.  Does it tell you more than _just_
that?  It doesn't for me.

> Unless someone wants to fix Windows,

As above, because removing the call to the internal `touch`
function makes all problems go away it's not obvious that
this is a Windows problem.

> a simple workaround would be to retry the os.remove a
> few times before giving up (with a time.sleep(0.1) in
> between).

Because of the internal threading errors in subprocess.py
(see my first comment), the threads in the test program
still usually die, but with instances of list.remove(x)
ValueErrors internal to subprocess.py.

If I hack around that, then this change to the test
program's file-removal code appears adequate to eliminate
all errors on my box (which is a zippy 3.4 GHz):

            try:
                os.remove(filename)
            except OSError:
                time.sleep(0.1)
                os.remove(filename)

It's possible that some virus-scanning or file-indexing
gimmick on my box is opening these little files for its own
purposes -- although, if so, I'm at a loss to account for
why a single "os.remove(filename)" never raises an exception
when the `touch()` call is commented out.

OTOH, with the `touch()` call intact, the time.sleep(0.1)
above is not adequate to prevent os.remove() errors if I
change the file-writing code to:

    f.write("test" * 250000)

Even boosting the sleep() to 0.4 isn't enough then.

That does (mildly) suggest there's another process opening
the temp files, and doing something with them that takes
time proportional to the file size.  However, the
os.remove() errors persist when I disable all such gimmicks
(that I know about ;-)) on my box.

It seems likely I'll never determine a cause for that.  The
bad thread behavior in subprocess.py is independent, and
should be repaired regardless.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2006-02-06 23:05

Message:
Logged In: YES 
user_id=38376

> The problem is that there's no reason to believe anything
> he did here _does_ leave files open.

Except that he's hitting the file system quite heavily, and
asyncronously.  My guess is that Windows simply gets behind
(a quick filemon test indicates that this is indeed the
case; just before a crash, I see the events CREATE/SUCCESS,
QUERY/SUCCESS, QUERY/SUCCESS, WRITE/SUCCESS, and
OPEN/SHARING VIOLATION for the failing file, with lots of
requests for other files interleaved).

Unless someone wants to fix Windows, a simple workaround
would be to retry the os.remove a few times before giving up
(with a time.sleep(0.1) in between).

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-02-06 22:53

Message:
Logged In: YES 
user_id=1276121

I did post on the python mailing list first, but got no 
responce there, after further looking into it, I seriously 
think there is at least one bug here.

here is the link to the post: 
http://mail.python.org/pipermail/python-list/2006-
February/323650.html

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2006-02-06 22:19

Message:
Logged In: YES 
user_id=31435

The problem is that there's no reason to believe anything he
did here _does_ leave files open.  I can confirm the
"permission denied" symptom, and even if I arrange for the
call to "touch" to run a touch.bat that doesn't even look at
the filename passed to it (let alone open or modify the file).

I also see a large number of errors of this sort:

Exception in thread Thread-8:
Traceback (most recent call last):
  File "C:\python24\lib\threading.py", line 442, in __bootstrap
    self.run()
  File "osremove.py", line 21, in run
    touch(filename)
  File "osremove.py", line 8, in touch
    stdin=None, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
  File "C:\python24\lib\subprocess.py", line 490, in __init__
    _cleanup()
  File "C:\python24\lib\subprocess.py", line 398, in _cleanup
    inst.poll()
  File "C:\python24\lib\subprocess.py", line 739, in poll
    _active.remove(self)
ValueError: list.remove(x): x not in list

Those are clearly due to subprocess.py internals on Windows,
where the poll() and wait() methods and the module internal
_cleanup() function aren't called in mutually threadsafe
ways.  _Those_ errors can be stopped by commenting out the
_cleanup() call at the start of Popen.__init__() (think
about what happens when multiple threads call _cleanup() at
overlapping times on Windows:  all those threads can end up
trying to remove the same items from _active, but only one
thread per item can succeed).

The "permission denied" errors persist, though.

So there's at least one class of subprocess.py Windows bugs
here, and another class of Windows mysteries.  I believe
subprocess.py is a red herring wrt the latter, though.  For
example, I see much the same if I use os.system() to run
`touch` instead.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2006-02-06 21:50

Message:
Logged In: YES 
user_id=38376

If Python gives you a permission error, that's because
Windows cannot remove the file.  Windows does, in general,
not allow you to remove files that are held open by some
process.

I suggest taking this issue to comp.lang.python.  The bug
tracker is not the right place for code review and other
support issues.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1425127&group_id=5470