[Python.NET] [PythonNet] 8/ 4 Resolve "Reentrancy bug"

[Collector: Python for .NET Issue ...]

Issue #8 Update (Resolve) "Reentrancy bug"
 Status Resolved, General/bug medium
To followup, visit:
  http://www.zope.org/Members/Brian/PythonNet/Collector/8

==============================================================
= Resolve - Entry #4 by Brian on Mar 31, 2005 1:21 pm

 Status: Pending => Resolved

Great - thanks for the followup.

I'm going to work on a 'tutorial for 
embedders' for the next release, as this 
isn't nearly well-documented enough...

-BL
________________________________________
= Comment - Entry #3 by Anonymous User on Mar 31, 2005 12:59 pm

You're right -- the bug is indeed caused by the lack of a lock acquisition in that code.  Thanks for the tip.  This item should be closed.

> = Comment - Entry #2 by Brian on Mar 31, 2005 9:45 am
> 
> Note that to be thread-safe and reentrancy-safe you 
> also need to make sure that any managed code you have 
> that uses Python.Runtime objects or methods is acquiring 
> the Python lock first. For example:
> 
> public class Example {
> 
>   public static int MethodThatCallsPython() {
>     // have to acquire the GIL to safely call back into
>     // Python!!
>     IntPtr state = PythonEngine.AcquireLock();
> 
>     PythonEngine.RunSimpleString("print 'hello'");
> 
>     // done using python for now, so release the GIL...
>     PythonEngine.ReleaseLock(state);
>   }
> 
> }
> 
> Forgetting to manage the lock around a callback into Python 
> is almost certain to cause a NullReferenceException (because 
> there will be no valid thread state when the callback is 
> made).
> 
> ________________________________________
> = Request - Entry #1 by Anonymous User on Mar 30, 2005 8:10 pm
> 
> There is a reentrancy problem when calling CLR code from Python that
>   calls back into Python.  The sequence I use to reproduce the problem is
>   fairly simple:
> 1) Execute some Python code either through PyObject.InvokeMethod, or 
> PythonEngine.RunSimpleString.
> 2) From the executed Python code, call into a CLR class.
> 3) From the CLR class, execute Python code again.
> 
> Here is some example code that demonstrates the issue:
> using System;
> using Python.Runtime;
> 
> namespace PDNBug {
> 	public class Example {
> 		[STAThread]
> 		static void Main(string[] args) {
> 			PythonEngine.Initialize();
> 			PythonEngine.RunSimpleString("from CLR.PDNBug import
>   			Example\nExample.FirstCall()");
> 		}
> 
> 		public static void FirstCall () {
> 			Console.WriteLine("First call succeeded.");
> 			PythonEngine.RunSimpleString("from CLR.PDNBug import
>   			Example\nExample.SecondCall()");
> 		}
> 
> 		public static void SecondCall () {
> 			Console.WriteLine("Second call succeeded.");
> 		}
> 	}
> }
> 
> The output generated is:
> First call succeeded.
> Traceback (most recent call last):
>   File "<string>", line 2, in ?
> System.NullReferenceException: Object reference not set to an instance of
>   an object.
>    at Python.Runtime.Runtime.PyRun_SimpleString(String code)
>    at Python.Runtime.PythonEngine.RunSimpleString(String code)
>    at PDNBug.Example.FirstCall() in example.cs:line 15: Object reference
>      not set to an instance of an object.
> 
> The expected output is:
> First call succeeded.
> Second call succeeded.
> 
> This issue was previously posted to the mailing list
>   (http://mail.python.org/pipermail/pythondotnet/2005-January/000235.html).
________________________________________
= Comment - Entry #2 by Brian on Mar 31, 2005 9:45 am

Note that to be thread-safe and reentrancy-safe you 
also need to make sure that any managed code you have 
that uses Python.Runtime objects or methods is acquiring 
the Python lock first. For example:

public class Example {

  public static int MethodThatCallsPython() {
    // have to acquire the GIL to safely call back into
    // Python!!
    IntPtr state = PythonEngine.AcquireLock();

    PythonEngine.RunSimpleString("print 'hello'");

    // done using python for now, so release the GIL...
    PythonEngine.ReleaseLock(state);
  }

}

Forgetting to manage the lock around a callback into Python 
is almost certain to cause a NullReferenceException (because 
there will be no valid thread state when the callback is 
made).

________________________________________
= Request - Entry #1 by Anonymous User on Mar 30, 2005 8:10 pm

There is a reentrancy problem when calling CLR code from Python that calls back into Python.  The sequence I use to reproduce the problem is fairly simple:
1) Execute some Python code either through PyObject.InvokeMethod, or 
PythonEngine.RunSimpleString.
2) From the executed Python code, call into a CLR class.
3) From the CLR class, execute Python code again.

Here is some example code that demonstrates the issue:
using System;
using Python.Runtime;

namespace PDNBug {
	public class Example {
		[STAThread]
		static void Main(string[] args) {
			PythonEngine.Initialize();
			PythonEngine.RunSimpleString("from CLR.PDNBug import Example\nExample.FirstCall()");
		}

		public static void FirstCall () {
			Console.WriteLine("First call succeeded.");
			PythonEngine.RunSimpleString("from CLR.PDNBug import Example\nExample.SecondCall()");
		}

		public static void SecondCall () {
			Console.WriteLine("Second call succeeded.");
		}
	}
}

The output generated is:
First call succeeded.
Traceback (most recent call last):
  File "<string>", line 2, in ?
System.NullReferenceException: Object reference not set to an instance of an object.
   at Python.Runtime.Runtime.PyRun_SimpleString(String code)
   at Python.Runtime.PythonEngine.RunSimpleString(String code)
   at PDNBug.Example.FirstCall() in example.cs:line 15: Object reference not set to an instance of an object.

The expected output is:
First call succeeded.
Second call succeeded.

This issue was previously posted to the mailing list (http://mail.python.org/pipermail/pythondotnet/2005-January/000235.html).
==============================================================