[Python-bugs-list] [ python-Bugs-454843 ] Python2.x sre and threaded import lockin

[noreply@sourceforge.net]

Bugs item #454843, was opened at 2001-08-23 20:40
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=454843&group_id=5470

Category: Threads
>Group: None
Status: Open
Resolution: None
>Priority: 4
Submitted By: Joe Kelly (jpkelly)
>Assigned to: Fredrik Lundh (effbot)
Summary: Python2.x sre and threaded import lockin

Initial Comment:
There appears to be a bug in the way Python2.x does 
import locking on
sre.  This is illustrated by the following code 
fragments.

$ cat retest
#!/usr/local/bin/python
__import__("ret")

$ cat ret.py
#!/usr/local/bin/python

import time, thread, string
#import time, thread
import re
#import pre
#re=pre

def mythread():
        time.sleep(2)
        url = "hello there"
        url = re.sub(" ", '', url)
        print "hello from threadland", url

thread.start_new_thread(mythread, ())

url = "hello main there"
url = re.sub(" ", '', url)
print "main", url
time.sleep(5)
print "goodbye", url


If you run retest as shown above you get:
$ retest
main hellomainthere
goodbye hellomainthere

It never runs the code in the thread.

If you run python on ret.pyc directly you get the 
correct/expected
result:
$ python ret.pyc
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

If you replace sre with pre by commenting out line 5 
and uncommenting
out line 6 and 7 it works correctly/as expected:
$ retest
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

I've also noticed some strange interaction with other 
imports, i.e. if
you go back to the original ret.py and comment out 
line 3 and
uncomment line 4 (just don't import the string module) 
you get the
following:
$ retest
main hellomainthere
goodbye hellomainthere
hello from threadland hellothere

That is the thread gets executed at the end or 
sometimes not at all:
$ retest
main hellomainthere
goodbye hellomainthere

Again, calling ret directly seems to work.
$ python ret.pyc
main hellomainthere
hello from threadland hellothere
goodbye hellomainthere

This has been tested on Solaris 8 running on Intel:
$ uname -a
SunOS machine 5.8 Generic_108529-07 i86pc i386 i86pc

It also exhibits the same behavior under Linux:
$ uname -a
Linux machine 2.4.2-2smp #1 SMP Sun Apr 8 20:21:34 EDT 
2001 i686 unknown

The above output was generated by python 2.0 compiled 
with threading
on:
$ python
Python 2.0 (#1, May 13 2001, 00:55:08)
[GCC 2.95.2 19991024 (release)] on sunos5
Type "copyright", "credits" or "license" for more 
information.
>>>

It exhibits the same behavior under python 2.1.1:
$ python
Python 2.1.1 (#22, Aug 20 2001, 17:12:35)
[GCC 2.95.2 19991024 (release)] on sunos5
Type "copyright", "credits" or "license" for more 
information.



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2001-08-23 21:20

Message:
Logged In: YES 
user_id=31435

Assigned to Fredrik, but as I said on c.l.py, never write a 
module that spawns threads as a side effect of merely being 
imported.  Everything you're chasing is an accident, and 
while /F may or may not bother worming around this one, 
you're going to get in trouble over and over again.  I've 
reduced the priority accordingly.

/F, the immediate cause for this one is the

        import sre_parse

in the body of sre_compile.compile(), which is reached in 
the spawned thread via calling re.sub().  Since the 
module "ret" is itself imported, the main thread doing the 
import of "ret" holds the import lock for the duration.  
This causes the spawned thread (created by merely 
importing "ret", and there's no proper synchronization in 
the test case either, so accidents compound) to block on 
any import attempt.  In particular, the spawned thread 
blocks on the line shown above.  Worm around that, and it 
blocks somewhere else later; etc.  In *general* it's 
hopeless.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=454843&group_id=5470