Mailman 3 October 2008 - capi-sig

ANNOUNCE: PyBindGen 0.8
by Gustavo Carneiro 10 Mar '22

10 Mar '22

The other day I released PyBindGen 0.8. Main news is that it features a new experimental header file scanner based on pygccxml (i.e., it's similar to py++ in scope, if not in maturity, but does not use boost.pythonunderneath). == What == PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write, and does not depend on any library or header files besides Python itself. == Where == https://launchpad.net/pybindgen/ == NEWS == - Support C++ instance attributes through getter/setter methods - Support functions as methods of C++ classes - Support the PyObject* type - Support unsigned int, C strings (char*) (from Mark Lee) - Add basic support for enum types - New experimental automatic module generator based on C/C++ header file scanner and annotations in comments, using pygccxml - Some bug fixes -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert

2 1

Passing non-PyObjects as args to PyObject_CallObject
by Sudharshan S 27 Oct '08

27 Oct '08

Hi all, My project uses GLib's GObject and recently I thought of adding support for executing python scripts by embedding the interpreter within the code. However there is this need to pass a GOB struct as one of the parameters to a function in the python script. How exactly do I pass non-PyObject types with the API? I tried (to be clever) casting my GOB structure to PyObject and passing it in the args field of PyObject_CallObject. This of course turned out to be an EPIC FAIL with a nasty segfault. (I also tried Py_BuildValue with 'O&' signature) It would be wonderful if some points me to some code which does this. From what I gather, do I need to create some sort of a Type Object which the Python API can understand? If it matters, I am using the 2.5 API. Thanks and regards Sudharshan S Blog : http://www.sudharsh.wordpress.com IRC : Sup3rkiddo @ Freenode, Gimpnet

2 1

Re: [capi-sig] Fix random crashes - embedding py/w PyObject pointers to app data.
by Adam Olsen 22 Oct '08

22 Oct '08

On Tue, Oct 21, 2008 at 10:54 PM, Campbell Barton <ideasman42(a)gmail.com> wrote: > On Wed, Oct 22, 2008 at 3:03 PM, Adam Olsen <rhamph(a)gmail.com> wrote: >> On Tue, Oct 21, 2008 at 9:34 PM, Campbell Barton <ideasman42(a)gmail.com> wrote: >>> Hi, just spent a couple of hours looking into a bug that others might also have. >>> If your app embeds python, and allows scripts to throw exceptions >>> while the application keeps running you may also have this problem. >>> >>> sys.last_traceback will store the last exception along with the >>> namespace dictionary of the script that just ran (with all its >>> functions etc, variables etc). >>> >>> In our case this caused crashes because you could load in new data, >>> making many of the PyObjects referenced from sys.last_traceback, point >>> to invalid data. Once sys.last_traceback was set again it would free >>> the existing data sometimes touching pointers with invalid addresses >>> causing a segfault. >>> >>> Simple solution is to run this after PyErr_Print(); >>> >>> PySys_SetObject( "last_traceback", Py_None); >>> >>> This can also free memory, if large datasets are references from the >>> traceback. maybe some of your would fine this useful. >> >> Do you mean you have a PyObject with a pointer to malloc'd memory, and >> after the exception is handled (but not cleared) you free the malloc'd >> memory, but leave the PyObject with a pointer to it? If so you should >> be keeping track of any PyObjects and clearing them instead (or let >> them take care of freeing the malloc'd memory). > > Yeah, this is the case and you're right, we should be aware of the > PyObjects existence and invalidate them when the memory is freed, or > let python do the freeing. > But our application does not center around python, the PyAPI has its > own directory and PyObjects are not supposed to be used outside that. > Id be interested in knowing what other apps do - is this even a common problem? Common in the sense that other apps probably do it right, yes. ;) How are you calling into python? If you call a single python function (which in turn calls many others), and this is what raises the exception, this is the obvious point to trigger the clearing. -- Adam Olsen, aka Rhamphoryncus

1 0

Fix random crashes - embedding py/w PyObject pointers to app data.
by Campbell Barton 22 Oct '08

22 Oct '08

Hi, just spent a couple of hours looking into a bug that others might also have. If your app embeds python, and allows scripts to throw exceptions while the application keeps running you may also have this problem. sys.last_traceback will store the last exception along with the namespace dictionary of the script that just ran (with all its functions etc, variables etc). In our case this caused crashes because you could load in new data, making many of the PyObjects referenced from sys.last_traceback, point to invalid data. Once sys.last_traceback was set again it would free the existing data sometimes touching pointers with invalid addresses causing a segfault. Simple solution is to run this after PyErr_Print(); PySys_SetObject( "last_traceback", Py_None); This can also free memory, if large datasets are references from the traceback. maybe some of your would fine this useful. -- - Campbell

2 1

Jack.Jansen@cwi.nl
by anti00Zero 21 Oct '08

21 Oct '08

Here is a screenshot and the call stack: ntdll.dll!_KiFastSystemCallRet@0() user32.dll!_NtUserWaitMessage@0() + 0xc bytes user32.dll!_InternalDialogBox@24() + 0xb6 bytes user32.dll!_SoftModalMessageBox@4() + 0x677 bytes user32.dll!_MessageBoxWorker@4() + 0x175 bytes user32.dll!_MessageBoxTimeoutW@24() + 0x7a bytes user32.dll!_MessageBoxTimeoutA@24() + 0x9c bytes user32.dll!_MessageBoxExA@20() + 0x1b bytes user32.dll!_MessageBoxA@16() + 0x45 bytes [Frames below may be incorrect and/or missing, no symbols loaded for python25.dll] kernel32.dll!_CreateFileA@28() + 0x2b bytes python26.dll!1e00f11b() python26.dll!1e00c7b5() python26.dll!1e007ac3() python26.dll!1e01cceb() msvcr90.dll!_sopen_helper(const char * path=0x0fd0dc68, int oflag=32768, int shflag=64, int pmode=384, int * pfh=0x0fd0d7c8, int bSecure=1) Line 190 + 0x5 bytes msvcr90.dll!_sopen_s(int * pfh=0x0fd0d7c8, const char * path=0x0fd0dc68, int oflag=32768, int shflag=64, int pmode=384) Line 907 + 0x16 bytes msvcr90.dll!_openfile(const char * filename=0x1e1a53b2, const char * mode=0x1e0097c7, int shflag=265345024, _iobuf * str=0x00000025) Line 272 + 0x17 bytes 00000025() python26.dll!1e0094af() python26.dll!1e01cf9d() msvcr90.dll!__lock_fhandle(int fh=89305032) Line 468 + 0x5 bytes NMMediaServer.dll!initMediaHomeWebInterface() Line 32 + 0x30 bytes C++ python26.dll!1e025c0d() python26.dll!1e020bc3() python26.dll!1e026cf5() python26.dll!1e026d27() ntdll.dll!_RtlGetFullPathName_Ustr@24() + 0x2f7 bytes python26.dll!1e00ed6e() python26.dll!1e009136() python26.dll!1e00fcc6() python26.dll!1e00919a() python26.dll!1e0157a1() ntdll.dll!_RtlAllocateHeapSlowly@12() + 0x3da bytes msvcr90.dll!malloc(unsigned int size=408) Line 163 + 0x5f bytes > python26.dll!1e01530b() python26.dll!1e010317() python26.dll!1e01fd3d() python26.dll!1e01d91e() python26.dll!1e020b31() msvcr90.dll!__lock_fhandle(int fh=3) Line 467 msvcr90.dll!_close(int fh=503378203) Line 62 + 0x8 bytes msvcr90.dll!_close(int fh=503378203) Line 66 + 0x5 bytes 0e439f10() python26.dll!1e010091() python26.dll!1e020b72() python26.dll!1e017284() python26.dll!1e0172bc() NMMediaServer.dll!std::_Iterator_base::~_Iterator_base() Line 151 + 0x28 bytes C++ NMUPnPServices.dll!CILibWebRequestHeader::CILibWebRequestHeader(packetheader * pHeader=0x00000000) Line 90 + 0x19 bytes C++ ffffffff() The problem is msvcr90.dll? Why this dll for Visual Studio 2008? I thought it should be 2005. :-? >I'm sorry. >Here are some facts: > >Winodws XP SP3 with newest updates >Visual Studio 2005 >Microsoft Visual C++ 2005 Redistributable I downloaded Python 2.6 installer for Windows from >python.org I use Python with boost 1.36 for Python (could this be a problem?) > >I want to use Python 2.6 because it was (i think so) built with VS2005. >So I haven't to use msvcr71.dll. >On monday I can send a stack trace and a screenshot of this crash a with E-Mail. > > >Jack Jansen wrote: >> This could be one of thousands of things. If it works with 2.5 and no >> more with 2.6 some possible candidates are (1) a bug in Python, (2) a >> difference in memory layout which caused buggy code to work correct in >> 2.5 nonetheless or (3) a timing difference, with the same effect. >> >> Oh yes, if you're on Windows and using the prebuilt dll it could also >> be caused by runtime library differences, I think 2.5 was built with >> VS2003 and 2.6 with VS2005. >> >> You'll need to provide a bit more detail if you want more help (OS and >> version, how you built Python and your app, is your code in an >> extension module or embedding Python, how does it crash, did you get a >> stack trace, etc) >> >> -- >> >> Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack >> <http://www.cwi.nl/%7Ejack> >> >> If I can't dance I don't want to be part of your revolution -- Emma >> Goldman >> >> >>

3 2

Python2.6 and PyImport_ImportModule
by anti00Zero 18 Oct '08

18 Oct '08

Hello. I've use Python 2.5 and want to use Python 2.6. But I have a problem with PyImport_ImportModule. c++ code: PyObject* pModule = PyImport_ImportModule(pBuffer); pBuffer is for example theme.simple.index and my script looks like: import MyModule import sys def StartWebInterface(cppObjectWeb, cppObjectSettings): cppObjectWeb.SetHTTPHeaderContentType(u'text/html; charset=utf-8') cppObjectWeb.SetHTTPHeaderExt(u'') page = u'' try: page = u'''<html> <head> <title>''' servername = cppObjectSettings.GetSettingsFromMethod(DeviceFriendlyName) page += servername page += u'''</title> </head> <body> <h1>''' page += servername page += u'''</h1> <a href="?f=browse">Browse Directory</a><br /> <a href="?f=settings">Settings</a> </body> </html>''' except: page = BuildErrorPage() cppObjectWeb.SetPage(page) def BuildErrorPage(): page = u'''<html> <head> <title>Python Error</title> </head> <body>''' page += sys.exc_info() page += u'</body></html>' return page But the PyImport_ImportModule crashed my c++ program. The index.py was compiled because a new index.pyc was created. Thanks for help. -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger

2 2

python and fpos_t
by Dorian Krause 09 Oct '08

09 Oct '08

Hello everybody, I stumbled about a curious problem: Including <Python.h> changes the size of fpos_t. m01% cat fpos_python.cc #ifdef HAVE_PYTHON_H #include <Python.h> #endif /* HAVE_PYTHON_H */ #include <stdio.h> int main(int argc, char** argv) { printf("sizeof(fpos_t) = %d\n",sizeof(fpos_t)); } m01% g++ -DHAVE_PYTHON_H -I/usr/include/python2.5 fpos_python.cc -lpython2.5 && ./a.out sizeof(fpos_t) = 16 m01% g++ -I/usr/include/python2.5 fpos_python.cc -lpython2.5 && ./a.out sizeof(fpos_t) = 12 This breaks my code if I do not include <Python.h> in ALL of my compilation modules (which is not pratical in my opinion). Is this a known problem? I personally would call this definitely a bug. What is the rational behind changing fpos_t? Thanks. Regards, Dorian K. ________________________________________________________________________ "50 erste Dates" mit Adam Sandler u. Drew Barrymore kostenlos anschauen! Exklusiv für alle WEB.DE Nutzer. http://www.blockbuster.web.de

3 2

overriding pythons import - change in 2.6
by Campbell Barton 03 Oct '08

03 Oct '08

was trying to hunt down why modules would not import in Blender3D with python 2.6, and found its caused by an extra argument to import that was making our own internal replacement for __builtin__.__import__ fail. Added an #ifdef to work around this, for anyone else replacing the builtin import, this could save you some time. static PyObject *blender_import( PyObject * self, PyObject * args ) { PyObject *exception, *err, *tb; char *name; PyObject *globals = NULL, *locals = NULL, *fromlist = NULL; PyObject *m; //PyObject_Print(args, stderr, 0); #if (PY_VERSION_HEX >= 0x02060000) int dummy_val; /* what does this do?*/ if( !PyArg_ParseTuple( args, "s|OOOi:bimport", &name, &globals, &locals, &fromlist, &dummy_val) ) return NULL; #else if( !PyArg_ParseTuple( args, "s|OOO:bimport", &name, &globals, &locals, &fromlist ) ) return NULL; #endif m = PyImport_ImportModuleEx( name, globals, locals, fromlist ); if( m ) return m; else PyErr_Fetch( &exception, &err, &tb ); /*restore for probable later use */ -- - Campbell

1 0

capi-sig October 2008