[Python-bugs-list] [Bug #128210] Threads using same stream blow up (Windows)

[noreply@sourceforge.net]

Bug #128210, was updated on 2001-Jan-09 13:10
Here is a current snapshot of the bug.

Project: Python
Category: Windows
Status: Open
Resolution: None
Bug Group: 3rd Party
Priority: 1
Submitted by: tim_one
Assigned to : nobody
Summary: Threads using same stream blow up (Windows)

Details: Blows up under released Windows 2.0 and CVS Pythons (so it's not
due to anything new):

import thread

def read(f):
    import time
    time.sleep(.01)
    n = 0
    while n < 1000000:
        x = f.readline()
        n += len(x)
        print "r",
    print "read " + `n`
    m.release()

m = thread.allocate_lock()
f = open("ga", "w+")
print "opened"
m.acquire()
thread.start_new_thread(read, (f,))
n = 0
x = "x" * 113 + "\n"
while n < 1000000:
    f.write(x)
    print "w",
    n += len(x)
m.acquire()
print "done"

Typical run:

C:\Python20>\code\python\dist\src\pcbuild\python temp.py
opened
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w r w r
w r w r w r w r w r w r w r w r w r w r w r w r w r w r w r w r w
r r w r w r w r w r w r

and then it dies in msvcrt.dll with a bad pointer.  Also dies under the
debugger (yay!) ... always dies like so:

+ We (Python) call the MS fwrite, from fileobject.c file_write.
+ MS fwrite succeeds with its _lock_str(stream) call.
+ MS fwrite then calls MS _fwrite_lk.
+ MS _fwrite_lk calls memcpy, which blows up for a non-obvious reason.

Looks like the stream's _cnt member has gone mildly negative, which
_fwrite_lk casts to unsigned and so treats like a giant positive count, and
so memcpy eventually runs off the end of the process address space.

Only thing I can conclude from this is that MS's internal stream-locking
implementation is buggy.  At least on W98SE.  Other flavors of Windows? 
Other platforms?

Note that I don't claim the program above is *sensible*, just that it
shouldn't blow up.  Alas, short of adding a separate mutex in Python file
objects-- or writing our own stdio --I don't believe I can fix this.


Follow-Ups:

Date: 2001-Jan-10 06:48
By: gvanrossum

Comment:
Mixing reads and writes on the same stream is always a bad idea. I believe
that the stdio docs prescribe that you must use fflush() whenever switching
between reading and writing. We could enforce this by adding two flags,
"reading" and "writing" to the file object.

(It's possible that using fseek() is also allowed to change directions.)

I'm not sure tht this is worth fixing though -- no sensible programmer will
do this.

-------------------------------------------------------

Date: 2001-Jan-10 00:37
By: tim_one

Comment:
Here's a standalone C program with the same symptom:

#include <process.h>
#include <stdio.h>

static FILE* fp;

void
reader(void* irrelevant)
{
	char buf[100];
	for (;;)  {
		char* p = fgets(buf, sizeof buf, fp);
		if (p) {
			putchar('r');
		}
	}
}

void
main()
{
	int i;
	char string[100];
	for (i = 0; i < sizeof(string) - 1; ++i) {
		string[i] = 'x';
	}
	string[sizeof(string) - 1] = '\n';

	fp = fopen("whatever", "w+");
	_beginthread(reader, 0, NULL);

	for (;;) {
		fwrite(string, 1, sizeof(string), fp);
		putchar('w');
	}
}

-------------------------------------------------------

Date: 2001-Jan-09 14:41
By: tim_one

Comment:
Adding info communicated via email:

+ Also blows up under Windows 2000.

+ Yes, C doesn't define what happens here even if it weren't threaded (you
can't mix reads and writes willy-nilly (without e.g. seeking before
reading) and expect something sensible).  The program is not sensible.  But
even silly programs shouldn't *blow up* in Python.

-------------------------------------------------------

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=128210&group_id=5470