[ python-Bugs-1163563 ] Sub threads execute in restricted mode

[SourceForge.net]

Bugs item #1163563, was opened at 2005-03-15 09:56
Message generated for change (Comment added) made by yetanothermax
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470

Category: Threads
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: anothermax (yetanothermax)
Assigned to: Nobody/Anonymous (nobody)
Summary: Sub threads execute in restricted mode

Initial Comment:
I'm using the JEP product which allows integration
of Java with Python (see http://jepp.sourceforge.net) via
starting a Python interpreter in the same process as the 
JVM.
This integrates with python via the C interface, allowing 
the user to 'eval' python code (amongst other features).

It seems that since Python 2.3.5 any threads (using the 
threading module) created via code that has been 
evaluated through the jep.eval() interface (i.e.Python C 
interface )are executed in restricted mode and cannot, 
for example, create files. Code that is just evaled (i.e not 
in a subthread) thread has no restrictions.

The error reported is:
IOError: file() constructor not accessible in restricted 
mode

(see http://sourceforge.net/forum/forum.php?
thread_id=1247793&forum_id=376782) - I've given a JEP 
specific example here.

There seems to be a similar problem with mod_python
(see 
http://www.modpython.org/pipermail/mod_python/2005-
January/017129.html)

Reading through the release notes for Python 2.3.5
I see:
 Bug #754449: threading.Thread will no longer mask 
exceptions raised during interpreter shutdown with 
another exception caused by attempting to output the 
initial exception. This fix also includes a backport of rev. 
1.41 from HEAD. 

This might be the problem as it seems to involve the 
porting of 2.4 threading code back to the 2.3 tree.

I've attached a Java file which shows the problem when 
using JEP.
The error output is:
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python24\Lib\threading.py", line 442, in 
__bootstrap
  File "<string>", line 5, in run
IOError: file() constructor not accessible in restricted 
mode

2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 
bit (Intel)]
Creating file from main thread...
Done
Creating file from sub thread...
Done


----------------------------------------------------------------------

>Comment By: anothermax (yetanothermax)
Date: 2005-04-20 13:39

Message:
Logged In: YES 
user_id=1218811

Here's the updated threadmodule.c for 2.4.1

----------------------------------------------------------------------

Comment By: anothermax (yetanothermax)
Date: 2005-04-20 13:36

Message:
Logged In: YES 
user_id=1218811

I've finally got access to a C compiler and it looks like 
the 'pure' Python changes for #754449 are in the clear 
(apologies to bcannon).

The problem actually seems to relate to changes to 
threadmodule.c of the Python Modules directory. 
In versions greater than 2.3.4 it uses the  PyGILState_XXX 
constructs t_bootstrap() function. 
These do not seem to be  compatible with running sub 
interpreters as JEP (and probably mod_python) do.
The python documentation 
(http://docs.python.org/api/threads.html) says "Note that the 
PyGILState_*() functions assume there is only one global 
interpreter (created automatically by Py_Initialize()). Python 
still supports the creation of additional interpreters (using 
Py_NewInterpreter()), but mixing multiple interpreters and the 
PyGILState_*() API is unsupported. "
I've compiled a new versions of the python2X.dll for both 2.3.5 
and 2.4.1 changing the use of these PyGIL_StateXXX 
constructs back to the 2.3.4 form and this seems to resolve 
the problem (I've attached the updated threadmodule.c files 
for both 2.3.5 and 2.4.1).


----------------------------------------------------------------------

Comment By: John Belmonte (jbelmonte)
Date: 2005-04-16 22:27

Message:
Logged In: YES 
user_id=282299

This problem also occurs on Linux.  Note however that   
I've traced the source of the symptom to a different bug  
fix, namely [1010677] thread Module Breaks  
PyGILState_Ensure()  
(http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470).   
Specifically, the change from v2.56 to v2.56.8.1 of  
threadmodule.c.  
  

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2005-04-16 21:21

Message:
Logged In: YES 
user_id=357491

To answer the comment on bug #754449 and the email I got
from someone, there is no way my fix for that bug would
cause this.  I only touched the 'threading' module, nothing
else.  That is in pure Python and thus has no direct way of
modifying what does and does not execute in restricted mode.
 If I understand how restricted execution works that is all
set at the C level.

Plus the fix only deals with the teardown code; it in no way
interfaces with how threads are executed.  All changed code
come after the thread and completed execution.  The only
thing it does ahead of time is store a reference to stdout.

In other words it ain't my fault to the best of my
knowledge.  =)

----------------------------------------------------------------------

Comment By: Alan Davies (goatpunch)
Date: 2005-03-31 08:24

Message:
Logged In: YES 
user_id=967140

The same problem definately does occur with mod_python, 
I've found it to occur with Python 2.3.5 and 2.4 on Win32:

http://www.modpython.org/pipermail/mod_python/2005-
March/017802.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470