[Python-bugs-list] [ python-Bugs-815646 ] thread unsafe file objects cause crash

[SourceForge.net]

Bugs item #815646, was opened at 2003-09-30 23:22
Message generated for change (Comment added) made by janixia
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=815646&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jan Olderdissen (janixia)
Assigned to: Nobody/Anonymous (nobody)
Summary: thread unsafe file objects cause crash

Initial Comment:
Note: This may be a dupe or a generalization of 595601.









Running below code snippet on 2.3.1 release and debug 

build on Windows 2000/XP a few times will inevitably lead 

to a crash. 2.2.2 also exhibits this behavior.









The crashing code:









------------





import thread









f=open("tmp1", "w")









def worker():





    global f





    while 1:





        f.close()





        f=open("tmp1", "w")





        f.seek (0, 0)









thread.start_new_thread(worker, ())





thread.start_new_thread(worker, ())





while 1: pass





-------------









The issue appears to be this (and similar) code sections 

from fileobject.c:









	Py_BEGIN_ALLOW_THREADS





	errno = 0;





	ret = _portable_fseek(f->f_fp, offset, whence);





	Py_END_ALLOW_THREADS









Note that due to 

Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS, 

f->f_fp can be set to NULL by file_close prior to entering 

_portable_fseek. Similar crashes can be observed with 

read, write and close itself when they are mixed with a 

concurrent close.









Obviously, the offending python code is buggy and a lock 

should be used to prevent concurrent access to f. 

However, it seems unreasonable for Python to crash 

because of it.









A mutex preventing concurrent access to the file object 

seems like a reasonable way to fix this.



----------------------------------------------------------------------

>Comment By: Jan Olderdissen (janixia)
Date: 2003-10-06 16:02

Message:
Logged In: YES 
user_id=877914

I'm inclined to go with Shane's suggested solution of 

reference counting when file access is in progress. It requires 

no synchronisation (increment, decrement and check are 

outside global lock release) and should have the smallest 

performance impact.



I don't think the FILE * extraction problem can be solved at 

all. Once the horse it out of the barn... However, for 

the "standard" case Shane's suggestion provides a neat and 

clean solution for the problem.



If the community can agree on this solution, I could be talked 

into implementing it.

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2003-10-05 20:48

Message:
Logged In: YES 
user_id=31392

Patch 595601 is an attempt to address this problem, but it's

incomplete.  The file object API allows an extension to

extract to FILE * and squirrel it away.  That's clearly

unsafe, because it can't participate in a locking scheme

without re-writing extensions.



Shane Hathaway proposed another solution here:

http://mail.python.org/pipermail/python-dev/2003-June/036543.html



The problem in this case is that we cause the call to

close() to raise an exception.  I'd prefer to see the

exception raised elsewhere, because close() seldom fails and

is often closed from routines that are cleaning up at the

end.  On the other hand, this solution would be easier to

implementation, so I'm at least +0 on it.



Let's do one or the other.



----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-10-04 01:01

Message:
Logged In: YES 
user_id=21627

janixia, don't worry about the formatting - this is

partially SF's fault, too.



Would you be interested in developing and testing a patch? I

think it would be sufficient to move the f->f_fp access out

of the GIL releasage.

----------------------------------------------------------------------

Comment By: Jan Olderdissen (janixia)
Date: 2003-10-01 00:08

Message:
Logged In: YES 
user_id=877914

My apologies for the mangled source. I suppose there isn't a 

way for me to remedy the situation.

----------------------------------------------------------------------

Comment By: Anthony Baxter (anthonybaxter)
Date: 2003-10-01 00:03

Message:
Logged In: YES 
user_id=29957

Attaching the failing code as an attachment, because SF

mangles source code.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=815646&group_id=5470