[Python.NET] Embedded Python .Net example - running scripts that import external libraries causes a crash

3 Oct 2018

      I am getting a crash trying to embed Python into my VB program using Python for .NET.  The goal of my application is to allow users to run their own python scripts inside a larger application.  The crash I am getting happens the second time I run a script that imports certain large libraries (like numpy or scikit-image) but does not affect all libraries (for example PIL).

My environment:
Windows 10 Pro x64, Python 3.6.5 (in Anaconda), Python for .Net 2.4.0 (installed from git’s master branch).

Here is the VB code:
    Public Function Startup(ByRef param As String) As Integer
        PythonEngine.Initialize()
        'mm.PrintMsg(PythonEngine.BuildInfo)
        'mm.PrintMsg(PythonEngine.Compiler)
        'mm.PrintMsg(PythonEngine.Platform)
        'mm.PrintMsg(PythonEngine.ProgramName)
        'mm.PrintMsg(PythonEngine.PythonHome)
        'mm.PrintMsg(PythonEngine.PythonPath)
        Using (Py.GIL())

            Dim scriptPath As String = param

            Dim scriptDir As String = Path.GetDirectoryName(scriptPath)
            Dim scriptName As String = GetPythonModuleName(scriptPath)
            Dim scriptParam As String = ""

            Dim scriptScope As PyScope = Py.CreateScope()

            Dim scriptText As String = My.Computer.FileSystem.ReadAllText(scriptPath)
            scriptScope.Exec(scriptText)

            If scriptScope.Contains(STARTUP_METHOD_NAME) Then
                scriptScope.Exec(String.Format("{0}(r'{1}')", STARTUP_METHOD_NAME, scriptParam))
            End If

            scriptScope.Dispose()
        End Using
        PythonEngine.Shutdown()

        Return 0
    End Function

And this is the Python script I am running:
import skimage.io

def Startup(param):
    pass

def Docommand(param):
    pass

def Shutdown():
    pass

When I run it the VB code the first time, things work great.  When I run the VB code a second time (without first restarting the entire application) I get an access violation exception on the scriptScope.Exec() call:
Exception: System.AccessViolationException :
Message: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Source: Python.Runtime
StackTrace:
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr kw)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyRun_String(String code, IntPtr st, IntPtr globals, IntPtr locals)
   at Python.Runtime.PyScope.Exec(String code, IntPtr _globals, IntPtr _locals)
   at MyApp.MyClass.Startup(String& param) in C:\Git\MyPath\MyApp.vb:line 132
 Unfortunately the stack trace doesn't show line numbers for anything in the Python.Runtime.

Interestingly, if you look at the VB code, I have some lines of commented out code:
        'mm.PrintMsg(PythonEngine.BuildInfo)
        'mm.PrintMsg(PythonEngine.Compiler)
        'mm.PrintMsg(PythonEngine.Platform)
        'mm.PrintMsg(PythonEngine.ProgramName)
        'mm.PrintMsg(PythonEngine.PythonHome)
        'mm.PrintMsg(PythonEngine.PythonPath)

Since the app doesn’t have a console, this sends the text to the enclosing app for display.  If I uncommon that code, then the first time I run it I get:
default, Mar 29 2018, 13:32:41
[MSC v.1900 64 bit (AMD64)]
win32
python

F:\Anaconda3\python36.zip;F:\Anaconda3\Lib;F:\Anaconda3\DLLs;C:\MX6

And the second time I run it I get:
default, Mar 29 2018, 13:32:41
[MSC v.1900 64 bit (AMD64)]
win32
pyth???
???
F:\Anaconda3\python36.zip;F:\Anaconda3\Lib;F:\Anaconda3\DLLs;C:\MX6

And the application doesn’t crash!  Instead it just says the skimage module can’t be found.  Notice the bolded lines (bold added after the fact).  The program name and python home values are being corrupted.
Is there something I am doing wrong as far as taking down the PythonEngine such that it could be put back up in an as-new state later on?

Steve

Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

[Python.NET] Embedded Python .Net example - running scripts that import external libraries causes a crash

Luke, Steve