[python-win32] Embedding python in a win32 dll and a wx thread

[mani sabri]

Hello everyone

 

It's a big post but I think it's an interesting problem and the result may
become a good recipe for embedding python in many applications in windows.

 

I'm on winxp sp2 using python 2.4 unicode, wxpython 2.8.7 unicode and
vs2003...

 

I'm embedding python in a non opensource financial application which only
lets me to import a dll in scripts which it calls experts. These experts
have three functions: 

 

1-init: which is called when you attach your expert to a chart. 

2-start: which is called whenever a new price (quote) is received.

3-deinit: which is called when you remove your expert from a chart.

 

For embedding python I thought of these 3 steps:

 

-In 'init' I call initpython and startshell from my dll which initializes
python[2], imports wx and starts a shell[3].

 

- In 'start' I send data to python for computation! (not relevant for now)

 

- In 'deinit' I exit wx app[4] and finalize python[5].

 

Everything works fine for the first time (python initialize/shell start/
shell terminate/ python finalize) But after a while I realized that I need
to reinitialize python and start all over again. 

 

The problem is that I have no control over loading and unloading my dll
Which means that the dll stays attached to the application process after
detaching from a chart and somehow it becomes the source of evil the second
time I try to attach it to a chart and I get this traceback[1] in the
pystartshell function when I try to create the wx app. 

 

Whats the problem? If as the traceback says I'm not in main thread why it
works the first time?

The application manual says it detaches the dll from the process when the
expert is removed from a chart. And I check the dllmain detach case and it
was true!!!

Is it the problem of some thread remaining active after finalizing python
(or dllmain detach)? 

How can I know if a thread remains alive after dllmain detach?

Is it usefull to fully unload the dll and the dlls the python loads when I
import wx from memory with win32 api freelibray? I tried it[6] without and
change in results, is it any other way to do it?

 

Any suggestion is welcomed, I'm a newbie, please criticize! I will be the
most grateful person on earth if you guide me!

 

Best regards,

Mani

 

Ps: I tried to attach a 5kb zip file containing the dll source and a mocking
application that produces the same error but I got mail delivery failure. If
you are the angle who is going to help me ;) let me know and I will send you
the source codes via email. 

 

[1]

Traceback (most recent call last):

  File "C:\Python24\lib\site-packages\wx-2.8-msw-unicode\wx\py\shell.py",
line 1416, in OnUpdateUI

    if id in (wx.ID_CUT, wx.ID_CLEAR):

AttributeError: 'NoneType' object has no attribute 'ID_CUT'

Traceback (most recent call last):

  File "C:\Python24\lib\site-packages\wx-2.8-msw-unicode\wx\_misc.py", line
1341, in Notify

    self.notify()

  File "C:\Python24\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line
14470, in Notify

    self.result = self.callable(*self.args, **self.kwargs)

  File "C:\Python24\lib\site-packages\wx-2.8-msw-unicode\wx\_windows.py",
line 1071, in SplitVertically

    return _windows_.SplitterWindow_SplitVertically(*args, **kwargs)

wx._core.PyAssertionError: C++ assertion "wxThread::IsMain()" failed at
..\..\src\msw\evtloop.cpp(244) in wxEventLoop::Dispatch(): only the main
thread can process Windows messages Traceback (most recent call last):

  File "C:\Python24\lib\site-packages\wx-2.8-msw-unicode\wx\py\shell.py",
line 1416, in OnUpdateUI

    if id in (wx.ID_CUT, wx.ID_CLEAR):

AttributeError: 'NoneType' object has no attribute 'ID_CUT'

 

 

[2]

EXPFUNC void __stdcall initPython(void)

{

      Py_Initialize();

      

      PyEval_InitThreads();

      m_mainThreadState = PyThreadState_Get();              

      

      PyEval_ReleaseLock();   

}

 

 

[3]

// this function is called in a separate thread 

Pystartshell(void *data)

{

      PyEval_AcquireLock();

      PyThreadState* pInterpreter   = Py_NewInterpreter();

 

      PyObject *globals = PyDict_New();

 

      PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins());


      PyDict_SetItemString(globals,"wx",PyImport_ImportModule("wx"));   

 

      PyRun_String("from wx.py import crust\n"

            "app = wx.PySimpleApp(redirect=False)\n"

            "frame = crust.CrustFrame(locals={'myApp':app})\n"

            "frame.Show()\n"

            ,Py_file_input,globals,globals);

 

      // app is global so I can exit this thread before finalizing python

      app = PyDict_GetItemString(globals,"app");

      Py_INCREF(app);

      PyObject_CallFunctionObjArgs(PyObject_GetAttrString(
app,"MainLoop"),NULL);

      

      Py_DECREF(globals);

      Py_EndInterpreter(pInterpreter);    

      PyEval_ReleaseLock();   

      _endthread();

}

 

 

[4]

//calling the exit function of wx application

void exit(void *a)

{

      PyEval_AcquireLock();

      PyThreadState* pInterpreter   = Py_NewInterpreter();

      PyObject_CallFunctionObjArgs(PyObject_GetAttrString(app,
"Exit"),NULL);

      Py_EndInterpreter(pInterpreter);    

      PyEval_ReleaseLock();         

      _endthread();

}

 

 

[5]

EXPFUNC void __stdcall deinitPython(void)

{     

 

      HANDLE ex = (HANDLE) _beginthread(exit,0,NULL);

      WaitForSingleObject(ex,INFINITE);

 

      PyEval_AcquireLock();

      Py_DECREF(app);

      

      PyThreadState_Swap(m_mainThreadState);

      Py_Finalize(); 

}

 

[6]

HINSTANCE  a1 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_co
re_.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a2 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\wxm
sw28uh_vc.dll", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a3 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_gd
i_.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a4 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_wi
ndows_.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a5 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_co
ntrols_.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a6 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_mi
sc_.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a7 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\_st
c.pyd", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

      HINSTANCE  a8 =
LoadLibraryEx("c:\\python24\\lib\\site-packages\\wx-2.8-msw-unicode\\wx\\wxm
sw28uh_stc_vc.dll", NULL,LOAD_WITH_ALTERED_SEARCH_PATH);

 

      FreeLibrary(a1);

      FreeLibrary(a2);

      FreeLibrary(a3);

      FreeLibrary(a4);

      FreeLibrary(a5);

      FreeLibrary(a6);

      FreeLibrary(a7);

      FreeLibrary(a8);

 

      FreeLibrary(a1);

      FreeLibrary(a2);

      FreeLibrary(a3);

      FreeLibrary(a4);

      FreeLibrary(a5);

      FreeLibrary(a6);

      FreeLibrary(a7);

      FreeLibrary(a8);

 

      a1=a2=a3=a4=a5=a6=a7=a8=NULL;

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-win32/attachments/20080612/3cca23c0/attachment-0001.htm>