[Python.NET] .NET 2 and hang with AcquireLock

[Matthew Pontefract]

Hi Brian

GenerateReport() is indeed called on a per-request basis but the python
engine is initialized only once: on subsequent calls
PythonEngine.IsInitialised returns true and the initialization code (all
of which is in the body of the if clause) is skipped (I've confirmed
this by stepping over the code).

Another thing that I tried as I was running out of ideas was to ensure
that the GenerateReport method was never run in parallel by locking the
entire code block, thus guaranteeing that there would only be one
attempt to acquire the GIL at any time (not that my testing ever
resulted in simultaneous calls to the function). Needless to say, this
did nothing to help matters!

Regards
Matthew

-----Original Message-----
From: Brian Lloyd [mailto:Brian.Lloyd at revolution.com] 
Sent: 06 June 2006 16:51
To: Matthew Pontefract; pythondotnet at python.org
Subject: RE: [Python.NET] .NET 2 and hang with AcquireLock

Thanks for the info. Looking at this, an idea came to mind 
that might be worth looking into:

Is GenerateReport() called on a per-request basis as a web 
service? If so, you might try initializing the python engine 
once at startup rather than per-request. I'm actually not sure 
that reinitializing the engine in the same process will 
work in any reliable way (there are allegations on the 
python lists that this is a grey area in regard to the 
C python runtime). 

If nothing else, it could be causing two instances of Reporting
to try to initialize the engine at the same time. Since they 
are both essentially initializing a single C runtime, I could 
imagine the GIL getting balled up in that case pretty easily.

-Brian


> -----Original Message-----
> From: Matthew Pontefract [mailto:MP at credaris.com] 
> Sent: Tuesday, June 06, 2006 11:37 AM
> To: Brian Lloyd; pythondotnet at python.org
> Subject: RE: [Python.NET] .NET 2 and hang with AcquireLock
> 
> Thanks for your reply Brian - I have been playing around with 
> this a fair bit. I checked out the source code from 
> Subversion; the revision from 2nd June built with no problems 
> and so I now have a .NET 2 build which appears to work well 
> from the console etc.
> 
> Using this from my C# web service exhibits the same behaviour 
> as before however! The problem is really quite tricky because 
> in fact it is wider than I previously suspected: the web 
> service will successfully acquire the lock once or twice in a 
> row, but will then hang at the AcquireLock call regardless of 
> whether a Python exception had been raised in code or not. 
> 
> I don't think any of the C# code would have been called again 
> (and I kept tabs on calls to acquire and drop the lock), nor 
> is there any re-entrant code. 
> 
> This prompted me to change my approach to a rather more 
> satisfactory one
> anyway: I use ZSI to create a separate web service from 
> python directly and call that web service from my C# app. I 
> use PythonNet because I want to use business code available 
> in .NET assemblies. Thanks to PythonNet it's been a breeze to 
> use Python in what is otherwise a very C# environment - I'm 
> really pleased. 
> 
> Even though I've changed tack, when I have time I'll get a 
> debugger onto the PythonNet code and try and trace this 
> further - I'll post back if I get anywhere. It would be 
> interesting to know if it's something peculiar to me!
> 
> For your information, as requested, the code where the 
> problem occurred is as follows (it's all a touch noddy - 
> various permutations of this were tried, so that's why the gs 
> lock object rather bizarrely ended up static etc...):
> 
> 	public class Reporting
> 	{
> 		private static IntPtr gs;
> 		public static void GenerateReport(String report)
> 		{
> 			if (!PythonEngine.IsInitialized)
> 			{
> 				PythonEngine.Initialize();
> 				gs = PythonEngine.AcquireLock();
> 				try
> 				{
> 	
> PythonEngine.RunSimpleString("import sys");
> 	
> PythonEngine.RunSimpleString("sys.path.append('g:/pyccolo')");
> 	
> PythonEngine.RunSimpleString("from report.server_init import *");
> 	
> PythonEngine.RunSimpleString("print(sys.path)");
> 	
> PythonEngine.RunSimpleString("o.flush()");
> 				}
> 				finally
> 				{
> 					PythonEngine.ReleaseLock(gs);
> 				}
> 			}
> 
> 			gs = PythonEngine.AcquireLock();
> 			try
> 			{
> 	
> PythonEngine.RunSimpleString("runReport('"+report+"')");
> 	
> PythonEngine.RunSimpleString("o.flush()");
> 			}
> 			finally
> 			{
> 				PythonEngine.ReleaseLock(gs);
> 			}
> 		}	
> 	}
> 
> Regards
> Matthew
> 
> -----Original Message-----
> From: Brian Lloyd [mailto:Brian.Lloyd at revolution.com]
> Sent: 06 June 2006 14:24
> To: Matthew Pontefract; pythondotnet at python.org
> Subject: RE: [Python.NET] .NET 2 and hang with AcquireLock
> 
> Hi Matthew - 
> 
> This is a tough kind of problem to debug through email ;)
> 
> Is there any chance that the exceptions you raise cause any 
> of your C# to be called again? 
> 
> If not, if there is any way you can distill your code to a 
> relatively small example that demonstrates the problem, send 
> it to me and I'll take a look.
> 
> -Brian
> 
> > -----Original Message-----
> > From: pythondotnet-bounces at python.org 
> > [mailto:pythondotnet-bounces at python.org] On Behalf Of Matthew 
> > Pontefract
> > Sent: Tuesday, June 06, 2006 6:08 AM
> > To: pythondotnet at python.org
> > Subject: [Python.NET] .NET 2 and hang with AcquireLock
> > 
> >  
> > 
> > Hello
> > 
> > 
> > I have a question pertaining to running Python NET under 
> .NET version 
> > 2 which, admittedly, is not the version for which the packages are 
> > built.
> > 
> >  
> > 
> > All works just fine - I am using the PythonEngine inside a web 
> > service, acquiring the GIL before operations, releasing it in a 
> > 'finally' block afterwards. This is fine provided no python 
> exceptions 
> > are raised in the Python code. Even though the GIL is 
> released in the 
> > C#, a subsequent call to AcquireLock will hang as if 
> waiting for it to 
> > be released. No calls to acquirelock have been made between 
> the last 
> > call to release the GIL and the call that hangs.
> > 
> > 
> > Has anyone else had this problem?
> > 
> >  
> > 
> > I am using Python 2.4
> > 
> >  
> > 
> > Regards
> > 
> > Matthew
> > 
> >  
> > 
> > Dr Matthew Pontefract
> > 
> > Credaris Portfolio Management 
> > 
> > ________________________________________
> > 
> >  
> > 
> > mp at credaris.com
> > 
> > T : (44) 207 925 8365
> > 
> > F : (44) 207 925 8399
> > 
> >  
> > 
> > CPM Advisers Limited
> > 
> > Norfolk House
> > 
> > 30 Charles II Street
> > 
> > London
> > 
> > SW1Y 4AE
> > 
> > www.credaris.com
> > 
> >  
> > 
> > Regulated by the Financial Services Authority
> > 
> >  
> > 
> > 
> 
> 
> http://www.credaris.com/disclaimer.php
> 
> 
> 


http://www.credaris.com/disclaimer.php