From lanyjie at yahoo.com Mon Feb 1 09:24:42 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Mon, 1 Feb 2010 00:24:42 -0800 (PST) Subject: [capi-sig] expy 0.5.1 released Message-ID: <511580.45863.qm@web54208.mail.re2.yahoo.com> Hi there, EXPY 0.5.1 released, the exception raising feature is enhanced so that you can raise any builtin exceptions. Custom exceptions will be supported soon. For more information, see http://expy.sourceforge.net/ EXPY is an expressway to extend Python! Regards, Yingjie From mal at egenix.com Mon Feb 1 11:27:21 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 01 Feb 2010 11:27:21 +0100 Subject: [capi-sig] Python 3.x PyObject_HEAD versus PyObject_VAR_HEAD In-Reply-To: <7E5ED525-E6CF-4CAA-ADD1-ED5FBDAD6551@semanchuk.com> References: <7E5ED525-E6CF-4CAA-ADD1-ED5FBDAD6551@semanchuk.com> Message-ID: <4B66AC89.8030105@egenix.com> Philip Semanchuk wrote: > Hi all, > I'm trying to port my posix_ipc extension to Python 3, and I'm confused > by the documentation for PyObject_HEAD and PyObject_VAR_HEAD. > > My understanding is that a PyObject-derived type should begin its > PyTypeObject struct with PyObject_HEAD and a PyVarObject-derived type > with PyObject_VAR_HEAD. the documentation says of PyVarObject, "This > type does not often appear in the Python/C API." > http://docs.python.org/3.1/c-api/structures.html#PyVarObject > > The doc also says that PyObject_VAR_HEAD, "is used when declaring new > types which represent objects with a length that varies from instance to > instance." > http://docs.python.org/3.1/c-api/structures.html#PyObject_VAR_HEAD > > I assumed, then, that PyObject_VAR_HEAD would also appear infrequently, > but that's not the case at all. In fact, I can't find PyObject_HEAD used > at all in the Python 3.1.1 source code, whereas PyObject_VAR_HEAD is > used hundreds of times, including for objects that don't have a length > (like the float type, for instance). I'm not sure how you searched for these, but I get somewhat different counts: 33 PyObject_HEAD 10 PyObject_VAR_HEAD in the Include/ dir of Python 3 (SVN) > Is the documentation wrong, or am I misreading it? Suggestions appreciated. The docs are correct. PyObject_HEAD does not include the field ob_size, PyObject_VAR_HEAD does. Perhaps you are confusing this with PyVarObject_HEAD_INIT() ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 01 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From philip at semanchuk.com Mon Feb 1 23:56:57 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Mon, 1 Feb 2010 17:56:57 -0500 Subject: [capi-sig] Python 3.x PyObject_HEAD versus PyObject_VAR_HEAD In-Reply-To: <4B66AC89.8030105@egenix.com> References: <7E5ED525-E6CF-4CAA-ADD1-ED5FBDAD6551@semanchuk.com> <4B66AC89.8030105@egenix.com> Message-ID: <2598EA8A-BA68-4EE6-AA0C-E933A52A28BE@semanchuk.com> On Feb 1, 2010, at 5:27 AM, M.-A. Lemburg wrote: > Philip Semanchuk wrote: >> Hi all, >> I'm trying to port my posix_ipc extension to Python 3, and I'm >> confused >> by the documentation for PyObject_HEAD and PyObject_VAR_HEAD. >> >> My understanding is that a PyObject-derived type should begin its >> PyTypeObject struct with PyObject_HEAD and a PyVarObject-derived type >> with PyObject_VAR_HEAD. the documentation says of PyVarObject, "This >> type does not often appear in the Python/C API." >> http://docs.python.org/3.1/c-api/structures.html#PyVarObject >> >> The doc also says that PyObject_VAR_HEAD, "is used when declaring new >> types which represent objects with a length that varies from >> instance to >> instance." >> http://docs.python.org/3.1/c-api/structures.html#PyObject_VAR_HEAD >> >> I assumed, then, that PyObject_VAR_HEAD would also appear >> infrequently, >> but that's not the case at all. In fact, I can't find PyObject_HEAD >> used >> at all in the Python 3.1.1 source code, whereas PyObject_VAR_HEAD is >> used hundreds of times, including for objects that don't have a >> length >> (like the float type, for instance). > > I'm not sure how you searched for these, but I get somewhat > different counts: > > 33 PyObject_HEAD > 10 PyObject_VAR_HEAD > > in the Include/ dir of Python 3 (SVN) > >> Is the documentation wrong, or am I misreading it? Suggestions >> appreciated. > > The docs are correct. > > PyObject_HEAD does not include the field ob_size, PyObject_VAR_HEAD > does. > > Perhaps you are confusing this with PyVarObject_HEAD_INIT() ?! THanks for your reply. You're right, I should have said that it is PyVarObject_HEAD_INIT that's used hundreds of times while PyObject_HEAD_INIT is hardly used at all (according to `grep -r`). Actually, I can state that even more strongly: PyObject_HEAD_INIT is only used once -- in the #define for PyVarObject_HEAD_INIT. After spending another hour reading the Python source, I think I get it now. As is often the case, the problem came from not reading carefully. When I saw this: PyTypeObject PyFloat_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "float", sizeof(PyFloatObject), I saw PyVarObject_HEAD_INIT being used to initialize a PyFloatObject, which is not the case -- it's initializing a PyTypeObject. Since the PyTypeObject struct definition uses PyObject_VAR_HEAD, it makes sense that PyVarObject_HEAD_INIT is used to initialize it. That also explains why PyVarObject_HEAD_INIT shows up so much in the Python source -- it's all those PyTypeObjects getting initialized. This brings up two more questions that I could probably answer with more reading, but if you have time for some further explanation I'd appreciate it. First, in what context would one use PyObject_HEAD_INIT? Second, why does the PyTypeObject struct definition use PyObject_VAR_HEAD when a type object has no concept of length? Cheers Philip From mal at egenix.com Tue Feb 2 12:54:06 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 02 Feb 2010 12:54:06 +0100 Subject: [capi-sig] Python 3.x PyObject_HEAD versus PyObject_VAR_HEAD In-Reply-To: <2598EA8A-BA68-4EE6-AA0C-E933A52A28BE@semanchuk.com> References: <7E5ED525-E6CF-4CAA-ADD1-ED5FBDAD6551@semanchuk.com> <4B66AC89.8030105@egenix.com> <2598EA8A-BA68-4EE6-AA0C-E933A52A28BE@semanchuk.com> Message-ID: <4B68125E.50407@egenix.com> Philip Semanchuk wrote: > > On Feb 1, 2010, at 5:27 AM, M.-A. Lemburg wrote: >> Perhaps you are confusing this with PyVarObject_HEAD_INIT() ?! > > > THanks for your reply. You're right, I should have said that it is > PyVarObject_HEAD_INIT that's used hundreds of times while > PyObject_HEAD_INIT is hardly used at all (according to `grep -r`). > Actually, I can state that even more strongly: PyObject_HEAD_INIT is > only used once -- in the #define for PyVarObject_HEAD_INIT. > > After spending another hour reading the Python source, I think I get it > now. As is often the case, the problem came from not reading carefully. > When I saw this: > > PyTypeObject PyFloat_Type = { > PyVarObject_HEAD_INIT(&PyType_Type, 0) > "float", > sizeof(PyFloatObject), Right, it's being used to init the (static) type object PyFloat_Type in the above case. > I saw PyVarObject_HEAD_INIT being used to initialize a PyFloatObject, > which is not the case -- it's initializing a PyTypeObject. Since the > PyTypeObject struct definition uses PyObject_VAR_HEAD, it makes sense > that PyVarObject_HEAD_INIT is used to initialize it. > > That also explains why PyVarObject_HEAD_INIT shows up so much in the > Python source -- it's all those PyTypeObjects getting initialized. Right. > This brings up two more questions that I could probably answer with more > reading, but if you have time for some further explanation I'd > appreciate it. First, in what context would one use PyObject_HEAD_INIT? > Second, why does the PyTypeObject struct definition use > PyObject_VAR_HEAD when a type object has no concept of length? It does: additional members can be stored in the variable part of the object, ie. low-level attributes which are exposed as slots in Python. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 02 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From lanyjie at yahoo.com Wed Feb 3 07:43:51 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Tue, 2 Feb 2010 22:43:51 -0800 (PST) Subject: [capi-sig] expy 0.5.2 released Message-ID: <857729.7198.qm@web54208.mail.re2.yahoo.com> Hi, expy is an expressway to extend python. in release 0.5.2, expy now supports custom exceptions, besides all built-in ones, and exception handling is made easy. for more info, see http://expy.sourceforge.net/ cheers, Yingjie From python_capi at behnel.de Wed Feb 3 08:50:06 2010 From: python_capi at behnel.de (Stefan Behnel) Date: Wed, 03 Feb 2010 08:50:06 +0100 Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <857729.7198.qm@web54208.mail.re2.yahoo.com> References: <857729.7198.qm@web54208.mail.re2.yahoo.com> Message-ID: <4B692AAE.1060707@behnel.de> Hi, Yingjie Lan, 03.02.2010 07:43: > in release 0.5.2, expy now supports custom exceptions, besides all built-in ones, and exception handling is made easy. How does that work? I.e., how do you handle exceptions that occur inside of the plain string that expy uses to integrate C code? Stefan From lanyjie at yahoo.com Wed Feb 3 10:46:28 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Wed, 3 Feb 2010 01:46:28 -0800 (PST) Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <4B692AAE.1060707@behnel.de> Message-ID: <105481.76240.qm@web54201.mail.re2.yahoo.com> Hi, > Yingjie Lan, 03.02.2010 07:43: > > in release 0.5.2, expy now supports custom exceptions, > besides all built-in ones, and exception handling is made > easy. > > How does that work? I.e., how do you handle exceptions that > occur inside of > the plain string that expy uses to integrate C code? > There are several ways to throw exceptions with expy. First of all, you can use @throws(...) to throw exceptions based on input and output of a function. Another possibility is to throw exceptions inside your function implementation, and upon return of your code, the wrapper function would return NULL after all necessary reference count management, if any exception is thrown by your function implementation code. Please also refer to the documentation of expy on exception handling here (it also includes how to create a custom exception, and how to manage exception inheritance): http://expy.sourceforge.net/tutorial.html#exception-for-arguments-returns Yingjie From python_capi at behnel.de Wed Feb 3 12:18:53 2010 From: python_capi at behnel.de (Stefan Behnel) Date: Wed, 03 Feb 2010 12:18:53 +0100 Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <105481.76240.qm@web54201.mail.re2.yahoo.com> References: <105481.76240.qm@web54201.mail.re2.yahoo.com> Message-ID: <4B695B9D.4010402@behnel.de> Yingjie Lan, 03.02.2010 10:46: >>> in release 0.5.2, expy now supports custom exceptions, >>> besides all built-in ones, and exception handling is made >>> easy. >> >> How does that work? I.e., how do you handle exceptions that >> occur inside of >> the plain string that expy uses to integrate C code? > > There are several ways to throw exceptions with expy. First of all, you > can use @throws(...) to throw exceptions based on input > and output of a function. Another possibility is to throw exceptions inside > your function implementation, and upon return of your code, the wrapper > function would return NULL after all necessary reference count management, > if any exception is thrown by your function implementation code. Please > also refer to the documentation of expy on exception handling here (it also > includes how to create a custom exception, and how to manage exception > inheritance): > > http://expy.sourceforge.net/tutorial.html#exception-for-arguments-returns Hmm, there's a lot of code in that example. Let's contrast that to Cython. ------------------------- MyError = gfield(exception('module1f.MyError', ValueError)) NegError = gfield(exception('module1f.NegEror', MyError)) ------------------------- Cython: class MyError(ValueError): pass class NegError(MyError): pass ------------------------- #'$return': returned value @function(int) @throws('$return<0', NegError, 'No negatives!') @throws('$return>100', ValueError, 'Out of range: %i', '$return') def range_check(n = int): return "n*n" ------------------------- Cython: def range_check(int n): cdef int retval = n*n if retval < 0: raise NegError('No negatives!') if retval > 100: raise ValueError('Out of range: %d' % retval) return retval Note that the C compiler will likely discard the "retval < 0" block here. I assume you meant to check the input value instead? ------------------------- @function(str) @throws('n<0', ValueError, 'No nagatives: %s=%i!','desc, n') @throws('$return==NULL', code=""" //you can also do sth else here PyErr_NoMemory(); """) def stringify(n=int, desc = str): return """{ char* s = malloc(sizeof(char)*20); if (s) sprintf(s, "%i", n); return s; }""" ------------------------- This example will actually leak memory. Also, it won't work correctly on a 64-bit system. Anyway, here's some Cython code for this: from stdio cimport sprintf from stdlib cimport malloc, free from python_exc cimport PyErr_NoMemory def stringify(int n, desc): cdef char* s if n < 0: raise ValueError('No negatives: %s = %d' % (desc,i)) s = malloc(sizeof(char)*20) if s is NULL: PyErr_NoMemory() sprintf(s, "%i", n) try: return s finally: free(s) Personally, I'd rather write this: def stringify(int n, str desc): if n < 0: raise ValueError('No negatives: %s = %d' % (desc,i)) return str(n) or even this: def stringify(unsigned int n): return str(n) which will also do the right thing. ------------------------- #throws(True/False, ...): #special case when you provide all code #for exception checking and handling #True for arguement, False for returned values @function(int) #argument checking @throws('den > 500', MyError, "Can't handle it!") @throws(True, code=""" int i = 2; while(i < den && (den % i)) i++; if (i != den){ //not a prime! PyErr_SetString(PyExc_Exception, "argument 'den' must be a prime!"); //usually don't return here //unless no argument is of object type, //as you need to DEC_REF it, which //is automatically handled later. } """) @throws(False, code=""" int i = 2; while(i < $return && ($return % i)) i++; if (i != $return){ //not a prime! PyErr_SetString(PyExc_Exception, "must return a prime!"); return NULL; } """) def prime_friend(den = int): return "den+2" ------------------------- Cython: cdef check_prime(int p, error_message): cdef int i for i in range(2, p): # I'd add at least a sqrt() here if not (p % i): raise Exception(error_message) def prime_friend(int den): cdef int i, retval if den > 500: raise MyError("Can't handle it!") check_prime(den) check_prime(den+2) return den+2 Looking at the two implementations, I actually wonder what the term "express way" means to you exactly. Stefan From lanyjie at yahoo.com Wed Feb 3 13:21:00 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Wed, 3 Feb 2010 04:21:00 -0800 (PST) Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <4B695B9D.4010402@behnel.de> Message-ID: <658733.81444.qm@web54207.mail.re2.yahoo.com> Hi, Thank you very much for making this fantastic contrast with Cython. Admittedly, Cython has a lot of good things, and it is improved a lot. The task expy is targeting at is different from that of Cython, and I hope expy provides more flexibility when it comes to extend Python. > >>> in release 0.5.2, expy now supports custom > exceptions, > >>> besides all built-in ones, and exception > handling is made > >>> easy. > >> > >> How does that work? I.e., how do you handle > exceptions that > >> occur inside of > >> the plain string that expy uses to integrate C > code? > > > > There are several ways to throw exceptions with expy. > First of all, you > > can use @throws(...) to throw exceptions based on > input > > and output of a function. Another possibility is to > throw exceptions inside > > your function implementation, and upon return of your > code, the wrapper > > function would return NULL after all necessary > reference count management, > > if any exception is thrown by your function > implementation code. Please > > also refer to the documentation of expy on exception > handling here (it also > > includes how to create a custom exception, and how to > manage exception > > inheritance): > > > > http://expy.sourceforge.net/tutorial.html#exception-for-arguments-returns > > > Hmm, there's a lot of code in that example. Let's contrast > that to Cython. > > ------------------------- > MyError = gfield(exception('module1f.MyError', > ValueError)) > NegError = gfield(exception('module1f.NegEror', MyError)) > ------------------------- > > Cython: > > ? ? class MyError(ValueError): pass > ? ? class NegError(MyError): pass > > Yes, this looks much easier, I need to modify the way exceptions are defined in expy, after which the same thing would be possible with expy. Thanks! There are many good things to learn from Cython. > ------------------------- > #'$return': returned value > @function(int) > @throws('$return<0', NegError, 'No negatives!') > @throws('$return>100', ValueError, 'Out of range: %i', > '$return') > def range_check(n = int): > ???return "n*n" > ------------------------- > > Cython: > > ? ? def range_check(int n): > ? ? ? ? cdef int retval = n*n > ? ? ? ? if retval < 0: > ? ? ? ? ? ? raise > NegError('No negatives!') > ? ? ? ? if retval > 100: > ? ? ? ? ? ? raise > ValueError('Out of range: %d' % retval) > ? ? ? ? return retval > > > Note that the C compiler will likely discard the "retval > < 0" block here. I > assume you meant to check the input value instead? > You are right, n*n < 0 would never happen. > > ------------------------- > @function(str) > @throws('n<0', ValueError, 'No nagatives: %s=%i!','desc, > n') > @throws('$return==NULL', > ???code=""" > ? ? //you can also do sth else here > ? ? PyErr_NoMemory(); > """) > def stringify(n=int, desc = str): > ???return """{ > ???char* s = malloc(sizeof(char)*20); > ???if (s) sprintf(s, "%i", n); > ???return s; > }""" > ------------------------- > > This example will actually leak memory. Also, it won't work > correctly on a > 64-bit system. First of all, thank you very much for pointing this out to me. I'm not sure if this is something unavoidable by the design of expy or if I can avoid this by improve my implementation code of this particular example. I will check this and see what I can do about it. Meanwhile, I wonder if you can give me some suggestions, especially on the memory leak issue. > > Anyway, here's some Cython code for this: > > ? ? from stdio cimport sprintf > ? ? from stdlib cimport malloc, free > ? ? from python_exc cimport PyErr_NoMemory > > ? ? def stringify(int n, desc): > ? ? ? ? cdef char* s > ? ? ? ? if n < 0: > ? ? ? ? ? ???raise > ValueError('No negatives: %s = %d' % (desc,i)) > ? ? ? ? s = malloc(sizeof(char)*20) > ? ? ? ? if s is NULL: > ? ? ? ? ? > ???PyErr_NoMemory() > ? ? ? ? sprintf(s, "%i", n) > ? ? ? ? try: > ? ? ? ? ? ???return > s > ? ? ? ? finally: > ? ? ? ? ? > ???free(s) > > Personally, I'd rather write this: > > ? ? def stringify(int n, str desc): > ? ? ? ? if n < 0: > ? ? ? ? ? ???raise > ValueError('No negatives: %s = %d' % (desc,i)) > ? ? ? ? return str(n) > > or even this: > > ? ? def stringify(unsigned int n): > ? ? ? ? return str(n) > > which will also do the right thing. > EXPY requires you to do things in pure C when it comes to logic/functionality, the bright side of this is that you have full control over the implementation. As I said, EXPY and Cython are different in their objectives. > > ------------------------- > #throws(True/False, ...): > #special case when you provide all code > #for exception checking and handling > #True for arguement, False for returned values > @function(int) #argument checking > @throws('den > 500', MyError, "Can't handle it!") > @throws(True, code=""" > ???int i = 2; > ???while(i < den && (den % i)) > i++; > ???if (i != den){ //not a prime! > ? ? ? PyErr_SetString(PyExc_Exception, > ? ? ? ???"argument 'den' must > be a prime!"); > ? ? ? //usually don't return here > ? ? ? //unless no argument is of object > type, > ? ? ? //as you need to DEC_REF it, which > ? ? ? //is automatically handled later. > ???} > """) > @throws(False, code=""" > ???int i = 2; > ???while(i < $return && ($return > % i)) i++; > ???if (i != $return){ //not a prime! > ? ? ? PyErr_SetString(PyExc_Exception, > ? ? ? ???"must return a > prime!"); > ? ? ? return NULL; > ???} > """) > def prime_friend(den = int): > ? ? return "den+2" > ------------------------- > > Cython: > > ? ? cdef check_prime(int p, error_message): > ? ? ? ? cdef int i > ? ? ? ? for i in range(2, p):? # > I'd add at least a sqrt() here > ? ? ? ? ? ? if not (p % i): > ? ? ? ? ? ? > ???raise Exception(error_message) > > ? ? def prime_friend(int den): > ? ? ? ? cdef int i, retval > ? ? ? ? if den > 500: > ? ? ? ? ? ? raise > MyError("Can't handle it!") > ? ? ? ? check_prime(den) > ? ? ? ? check_prime(den+2) > ? ? ? ? return den+2 > > > Looking at the two implementations, I actually wonder what > the term > "express way" means to you exactly. > Well, in EXPY, you can also define this check_prime(.) function and then use it, I will improve this the way you did. Thanks again! Yingjie From python_capi at behnel.de Wed Feb 3 14:08:51 2010 From: python_capi at behnel.de (Stefan Behnel) Date: Wed, 03 Feb 2010 14:08:51 +0100 Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <658733.81444.qm@web54207.mail.re2.yahoo.com> References: <658733.81444.qm@web54207.mail.re2.yahoo.com> Message-ID: <4B697563.1010805@behnel.de> Yingjie Lan, 03.02.2010 13:21: > The task expy is targeting at is different from that of Cython Well, expy generates plain glue code, whereas Cython is a full fledged programming language with an optimising compiler. Both aim to simplify writing extension modules for CPython, and both interface with C code. It's not the task that's different, it's the feature set. > and I hope expy provides more > flexibility when it comes to extend Python. I guess I'm too biased to comment on that one. Stefan From lanyjie at yahoo.com Thu Feb 4 01:15:59 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Wed, 3 Feb 2010 16:15:59 -0800 (PST) Subject: [capi-sig] expy 0.5.2 released In-Reply-To: <4B697563.1010805@behnel.de> Message-ID: <484106.37926.qm@web54201.mail.re2.yahoo.com> > I guess I'm too biased to comment on that one. Not at all, I think what you have pointed out helped improve expy. I have corrected the memory leak and 64-bit unsafeness in my example. Thanks again, Yingjie From gjcarneiro at gmail.com Sun Feb 7 19:40:06 2010 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 7 Feb 2010 18:40:06 +0000 Subject: [capi-sig] ANN: PyBindGen 0.14 Message-ID: PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write. It can be downloaded from: http://code.google.com/p/pybindgen/ Bug reports should be filed here: https://bugs.launchpad.net/ pybindgen Documentation: http://packages.python.org/PyBindGen/ NEWS: === pybindgen 0.14 === - Multiple inheritance support - Virtual methods can now be overridden with Method instead of _Method - Add annotation support for instance attributes - Benchmarks (vs Boost.Python, SWIG, and SIP) - New types supported: int16_t& and std::string* parameter types - Non-virtual protected methods are now also wrapped - Wrap enum pointer params Note: this release marks the beginning of a transition to a minor change in the API of the generated bindings. Prior to PyBindGen version 0.14, the code generated to handle C++ virtual methods required Python user code to define a _foo method in order to implement the virtual method foo. Since 0.14, PyBindGen changed so that virtual method foo is implemented in Python by defining a method foo, i.e. no underscore prefix is needed anymore. Setting pybindgen.settings.deprecated_virtuals to True will force the old virtual method behaviour. But this is really deprecated; newer code should set pybindgen.settings.deprecated_virtuals to False. In PyBindGen 0.14, if the option pybindgen.settings.deprecated_virtuals is not set, PyBindGen emits a warning and assumes the value of True in order to preserve backward compatibility. In PyBindGen 0.15, the default value of this option will change to False, and in 0.16 the support for deprecated virtuals will be removed. This change was made to make the user code interface more intuitive, and to align it with other Python bindings tools such as SIP, Boost.Python, and SWIG. -- Gustavo J. A. M. Carneiro INESC Porto, UTM, WiN, http://win.inescporto.pt/gjc "The universe is always one step beyond logic." -- Frank Herbert From lanyjie at yahoo.com Thu Feb 11 04:40:54 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Wed, 10 Feb 2010 19:40:54 -0800 (PST) Subject: [capi-sig] expy 0.6 released Message-ID: <155225.21818.qm@web54203.mail.re2.yahoo.com> Hi: EXPY (http://expy.sourceforge.net/) is an express way to extend Python. Why consider expy? Here are some good reasons: (I). WYSIWYG. The expy project enables you to write your module in Python the way your extension would be (WYSIWYG), and meanwhile write your implementation in pure C. You specify your modules, functions, methods, classes, and even their documentations the usual way of writing your Python correspondences. Then provide your implementation to the functions/methods by returning a multi-line string. By such an arrangement, everything falls in its right place, and your extension code becomes easy to read and maintain. Also, the generated code is very human-friendly. (II). You only provide minimal information to indicate your intension of how your module/class would function in Python. So your extension is largely independent from the Python extension API. As your interaction with the Python extension API is reduced to minimal (you only care about the functionality and logic), it is then possible that your module written in expy can be independent of changes in the extension API. (III). The building and setup of your project can be automatically done with the distutil tool. In the tutorial, there are ample examples on how easily this is achieved. (IV). Very light weight. The expy tool is surprisingly light weight dispite of its powerful ability, as it is written in pure Python. There is no parser or compiler for code generation, but rather the powerful reflexion mechanism of Python is exploited in a clever way to generate human-friendly codes. Currently, generating code in C is supported, however, the implementation is well modularized and code generation in other languages such as Java and C++ should be easy. Currently, EXPY/CXPY works with Python 2.5, 2.6. I haven?t tested with Python 2.7 or 3.x (for 3.x, need to use the code tool first, and I am not sure if the C API for 3.x has changed much, if you happen to try it out, let me know what happened). While there are already a couple of other projects trying to simply this task with different strategies, such as Cython, Pyrex and modulator, this project is unique and charming in its own way. All you need is the WYSIWYG Python file for your module extension, then expy takes care of everything else. What follows in this documentation is on how to extend Python in C using expy-cxpy: the module expy helps define your module, while module cxpy helps generate C codes for your defined module. Yingjie From guillaume.yziquel at citycable.ch Sun Feb 14 14:44:42 2010 From: guillaume.yziquel at citycable.ch (Guillaume Yziquel) Date: Sun, 14 Feb 2010 14:44:42 +0100 Subject: [capi-sig] How to inspect module objects. Message-ID: <4B77FE4A.5090208@citycable.ch> Hello. I'm currently reworking the existing software embedding Python (v2.5 for now) into Objective Caml. You can already do some trivial things with it: > yziquel at seldon:~$ ocaml > Objective Caml version 3.11.1 > > # #use "topfind";; > - : unit = () > Findlib has been successfully loaded. Additional directives: > #require "package";; to load a package > #list;; to list the available packages > #camlp4o;; to load camlp4 (standard syntax) > #camlp4r;; to load camlp4 (revised syntax) > #predicates "p,q,...";; to set these predicates > Topfind.reset();; to force that packages will be reloaded > #thread;; to enable threads > > - : unit = () > # #require "python.interpreter";; > /usr/lib/ocaml/python: added to search path > /usr/lib/ocaml/python/python.cma: loaded > /usr/lib/ocaml/python/oCamlPython.cmo: loaded > # let q = Python.Module.import "dolfin";; > val q : Python.obj = > # let w = Python.Module.import "modulethatdoesnotexist";; > Exception: Python.Exception.Python_failure . > # So it catches exceptions when trying to import a module that does not exist, and it succeeds when importing dolfin, a module that does exist on my box. But then I'd like to inspect the contet of this module. So, here's my question: How do you access the content of this module, or the content of the python object, preferably with the C API. All apologies if these questions seem trivial to you, but I'm quite new to python and its internals. Pointers to relevant (tutorial?) documentation are appreciated. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/ From mal at egenix.com Sun Feb 14 17:14:15 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 14 Feb 2010 17:14:15 +0100 Subject: [capi-sig] How to inspect module objects. In-Reply-To: <4B77FE4A.5090208@citycable.ch> References: <4B77FE4A.5090208@citycable.ch> Message-ID: <4B782157.9050800@egenix.com> Guillaume Yziquel wrote: > Hello. > > I'm currently reworking the existing software embedding Python (v2.5 for > now) into Objective Caml. > > You can already do some trivial things with it: > >> yziquel at seldon:~$ ocaml >> Objective Caml version 3.11.1 >> >> # #use "topfind";; >> - : unit = () >> Findlib has been successfully loaded. Additional directives: >> #require "package";; to load a package >> #list;; to list the available packages >> #camlp4o;; to load camlp4 (standard syntax) >> #camlp4r;; to load camlp4 (revised syntax) >> #predicates "p,q,...";; to set these predicates >> Topfind.reset();; to force that packages will be reloaded >> #thread;; to enable threads >> >> - : unit = () >> # #require "python.interpreter";; >> /usr/lib/ocaml/python: added to search path >> /usr/lib/ocaml/python/python.cma: loaded >> /usr/lib/ocaml/python/oCamlPython.cmo: loaded >> # let q = Python.Module.import "dolfin";; >> val q : Python.obj = >> # let w = Python.Module.import "modulethatdoesnotexist";; >> Exception: Python.Exception.Python_failure . >> # > > So it catches exceptions when trying to import a module that does not > exist, and it succeeds when importing dolfin, a module that does exist > on my box. > > But then I'd like to inspect the contet of this module. So, here's my > question: How do you access the content of this module, or the content > of the python object, preferably with the C API. > > All apologies if these questions seem trivial to you, but I'm quite new > to python and its internals. Pointers to relevant (tutorial?) > documentation are appreciated. Some pointers: http://docs.python.org/c-api/module.html#PyModule_GetDict http://docs.python.org/c-api/dict.html The C-API documentation should help you dig further into Python objects: http://docs.python.org/c-api/index.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 14 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From guillaume.yziquel at citycable.ch Sun Feb 14 17:39:12 2010 From: guillaume.yziquel at citycable.ch (Guillaume Yziquel) Date: Sun, 14 Feb 2010 17:39:12 +0100 Subject: [capi-sig] How to inspect module objects. In-Reply-To: <4B782157.9050800@egenix.com> References: <4B77FE4A.5090208@citycable.ch> <4B782157.9050800@egenix.com> Message-ID: <4B782730.4010303@citycable.ch> M.-A. Lemburg a ?crit : >> >> All apologies if these questions seem trivial to you, but I'm quite new >> to python and its internals. Pointers to relevant (tutorial?) >> documentation are appreciated. > > Some pointers: > > http://docs.python.org/c-api/module.html#PyModule_GetDict > http://docs.python.org/c-api/dict.html Thanks. Exactly what I was looking for. > The C-API documentation should help you dig further into > Python objects: > > http://docs.python.org/c-api/index.html I already knew about this C API documentation. But getting familiar quickly with such a documentation is not an easy task. Thank you for your help. -- Guillaume Yziquel http://yziquel.homelinux.org/ From guillaume.yziquel at citycable.ch Sun Feb 14 22:41:46 2010 From: guillaume.yziquel at citycable.ch (Guillaume Yziquel) Date: Sun, 14 Feb 2010 22:41:46 +0100 Subject: [capi-sig] How to inspect module objects. In-Reply-To: <4B782157.9050800@egenix.com> References: <4B77FE4A.5090208@citycable.ch> <4B782157.9050800@egenix.com> Message-ID: <4B786E1A.2040201@citycable.ch> M.-A. Lemburg a ?crit : > > Some pointers: > > http://docs.python.org/c-api/module.html#PyModule_GetDict > http://docs.python.org/c-api/dict.html > > The C-API documentation should help you dig further into > Python objects: > > http://docs.python.org/c-api/index.html Here's a question on which I find little information on the web. About Py_ssize_t. First, some quick background about OCaml values: OCaml values are essentially of two sorts. There are integers, or integer-like OCaml values. And pointers of blocks in the OCaml heap. Both live in the same 32-bit or 64-bit type, called "value". Blocks in the OCaml heap are aligned, and the lowest 2 or 3 bits of a pointer are therefore always nulled. Taking advantage of this, an OCaml integer always ends with a lowest bit set to 1. To get the real / semantic integer, you just do a >> 1. And therefore OCaml ints are stored as 31 bits or 63 bits, depending on your architecture. The lowest bit thingy is used for OCaml's garbage collector, in order to know whether or not it should follow the value/pointer when doing its job traversing the OCaml heap. So my question is the following: is there any alignment voodoo in Python that would imply that Py_ssize_t values have their lowest bit cleared? If yes, I could map it to an integral OCaml type by adding 1. (The GC would not follow the value/pointer). If not, I have no other choice but to box the Py_ssize_t value. Thoughts welcome. -- Guillaume Yziquel http://yziquel.homelinux.org/ From mal at egenix.com Mon Feb 15 14:51:54 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 15 Feb 2010 14:51:54 +0100 Subject: [capi-sig] How to inspect module objects. In-Reply-To: <4B786E1A.2040201@citycable.ch> References: <4B77FE4A.5090208@citycable.ch> <4B782157.9050800@egenix.com> <4B786E1A.2040201@citycable.ch> Message-ID: <4B79517A.7020906@egenix.com> Guillaume Yziquel wrote: > M.-A. Lemburg a ?crit : >> >> Some pointers: >> >> http://docs.python.org/c-api/module.html#PyModule_GetDict >> http://docs.python.org/c-api/dict.html >> >> The C-API documentation should help you dig further into >> Python objects: >> >> http://docs.python.org/c-api/index.html > > Here's a question on which I find little information on the web. About > Py_ssize_t. > > First, some quick background about OCaml values: OCaml values are > essentially of two sorts. There are integers, or integer-like OCaml > values. And pointers of blocks in the OCaml heap. Both live in the same > 32-bit or 64-bit type, called "value". > > Blocks in the OCaml heap are aligned, and the lowest 2 or 3 bits of a > pointer are therefore always nulled. Taking advantage of this, an OCaml > integer always ends with a lowest bit set to 1. To get the real / > semantic integer, you just do a >> 1. And therefore OCaml ints are > stored as 31 bits or 63 bits, depending on your architecture. > > The lowest bit thingy is used for OCaml's garbage collector, in order to > know whether or not it should follow the value/pointer when doing its > job traversing the OCaml heap. > > So my question is the following: is there any alignment voodoo in Python > that would imply that Py_ssize_t values have their lowest bit cleared? No, Py_ssize_t values can use the full (signed) range provided by the underlying type. They are used for positive-only values such as length information as well as positive and negative values such as indexes. In some cases -1 is used to flag an error for APIs that normally only return non-negative values. > If yes, I could map it to an integral OCaml type by adding 1. (The GC > would not follow the value/pointer). If not, I have no other choice but > to box the Py_ssize_t value. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 15 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From guillaume.yziquel at citycable.ch Sat Feb 20 02:57:33 2010 From: guillaume.yziquel at citycable.ch (Guillaume Yziquel) Date: Sat, 20 Feb 2010 02:57:33 +0100 Subject: [capi-sig] varargs in the C API. Message-ID: <4B7F418D.4020006@citycable.ch> Hello. I've been trying to wrap up the C functions that have an ellipsis, or a vararg, such as PyTuple_Pack. I tried to use avcall.h to pass a dynamically-determined number of arguments to the underlying function. Having a hard time: > ocaml -init ocamlinit > Objective Caml version 3.11.1 > > # let x = ((Python.of_int 6) :> Python.obj);; > val x : Python.obj = > # let y = ((Python.of_int 4) :> Python.obj);; > val y : Python.obj = > # let args = Python.tuple_pack [x; y];; > [seldon:09731] *** Process received signal *** > [seldon:09731] Signal: Segmentation fault (11) > [seldon:09731] Signal code: Address not mapped (1) > [seldon:09731] Failing at address: 0x5 > [seldon:09731] [ 0] /lib/libpthread.so.0 [0x2b9099aca990] > [seldon:09731] [ 1] /usr/lib/libavcall.so(__builtin_avcall+0x2a5) [0x2b909a14c905] > [seldon:09731] *** End of error message *** > make: *** [test] Erreur de segmentation So just wondering: has anyone succeeded in wrapping functions like PyTuple_Pack using stuff like avcall, libffi, or cinvoke? Here's the code of my wrapper: > CAMLprim value ocamlpython_py_tuple_pack (value ml_len, value ml_pyobjs) { > > av_alist argList; > PyObject * retVal; > av_start_ptr(argList, PyTuple_Pack, PyObject*, retVal); > > #if defined(__s390__) || defined(__hppa__) || defined(__cris__) > #define av_Py_ssize_t av_long > #else > #define av_Py_ssize_t av_int > #endif > > av_Py_ssize_t(argList, Pyoffset_val(ml_len)); > while (ml_pyobjs != Val_emptylist) { > av_ptr(argList, PyObject*, Pyobj_val(Field(ml_pyobjs, 0))); > ml_pyobjs = Field(ml_pyobjs, 1); > } > > av_call(argList); > return(Val_owned_pyobj(retVal)); > } Please tell me if you see something fishy that my eyes do not catch. -- Guillaume Yziquel http://yziquel.homelinux.org/ From guillaume.yziquel at citycable.ch Sat Feb 20 03:28:51 2010 From: guillaume.yziquel at citycable.ch (Guillaume Yziquel) Date: Sat, 20 Feb 2010 03:28:51 +0100 Subject: [capi-sig] varargs in the C API. In-Reply-To: <4B7F418D.4020006@citycable.ch> References: <4B7F418D.4020006@citycable.ch> Message-ID: <4B7F48E3.3010003@citycable.ch> Guillaume Yziquel a ?crit : > Hello. > > So just wondering: has anyone succeeded in wrapping functions like > PyTuple_Pack using stuff like avcall, libffi, or cinvoke? > > Here's the code of my wrapper: Stupid me: >> av_start_ptr(argList, PyTuple_Pack, PyObject*, retVal); was wrong. > av_start_ptr(argList, &PyTuple_Pack, PyObject*, &retVal) was the correct invocation. -- Guillaume Yziquel http://yziquel.homelinux.org/ From Girish.Goudar at goodrich.com Wed Feb 24 11:13:07 2010 From: Girish.Goudar at goodrich.com (Goudar, Girish) Date: Wed, 24 Feb 2010 18:13:07 +0800 Subject: [capi-sig] (no subject) Message-ID: Hi All, I want to pass a structure (which contains int, float, char etc) from a client program written in Python to a server program written in C (Operating system is DEOS). But the "socket.send" command in Python supports only String or Readonly buffer. I want to pass a structure. Could you please help me on this? Thanks and Regards, Girish P.G From python_capi at behnel.de Wed Feb 24 12:40:01 2010 From: python_capi at behnel.de (Stefan Behnel) Date: Wed, 24 Feb 2010 12:40:01 +0100 Subject: [capi-sig] (no subject) In-Reply-To: References: Message-ID: <4B851011.2030008@behnel.de> Goudar, Girish, 24.02.2010 11:13: > I want to pass a structure (which contains int, float, char etc) from a > client program written in Python to a server program written in C > (Operating system is DEOS). But the "socket.send" command in Python > supports only String or Readonly buffer. I want to pass a structure. > Could you please help me on this? This has nothing to do with the Python C-API (so this is the wrong list to ask this on). What you have to do is figure out the structure of the data stream that the server requires, and then send your data in that structure. Stefan From hniksic at xemacs.org Wed Feb 24 13:05:17 2010 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Wed, 24 Feb 2010 13:05:17 +0100 Subject: [capi-sig] (no subject) In-Reply-To: (Girish Goudar's message of "Wed, 24 Feb 2010 18:13:07 +0800") References: Message-ID: <87bpfedeki.fsf@busola.homelinux.net> "Goudar, Girish" writes: > I want to pass a structure (which contains int, float, char etc) from > a client program written in Python to a server program written in C > (Operating system is DEOS). But the "socket.send" command in Python > supports only String or Readonly buffer. I want to pass a structure. > Could you please help me on this? Aside from the remark brought up by Stefan, I'd recommend the use of the "struct" module. From hniksic at xemacs.org Fri Feb 26 18:11:36 2010 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Fri, 26 Feb 2010 18:11:36 +0100 Subject: [capi-sig] (no subject) In-Reply-To: (Girish Goudar's message of "Thu, 25 Feb 2010 22:37:49 +0800") References: Message-ID: <87iq9j3os7.fsf@busola.homelinux.net> "Goudar, Girish" writes: > Thanks for the quick reply. I can use the struct module at Python side > and use the pack() function to pack the data. But in the DEOS side I > need to unpack the data for that I need to use unpack() function. But > DEOS is not supporting unpack() function. What to do? I meant to use that you can use struct.pack to create data that can be interpreted as a structure defined in C, assuming the same architecture is run on both machines. Such sharing of data is one of the use cases of the struct module. For example: # python side: import struct s = struct.pack('cid', 'A', 10, 20.0) send_data_to_network(s) /* C side: */ struct data_desc { char c; int i; double d; }; char *s; struct data_desc data; s = read_data_from_network(); memcpy(&data, s, sizeof(data)); /* data.c is now the char data.i the int data.d the float (double) */