capi-sig December 2008

capi-sig@python.org

6 participants
7 discussions

by Campbell Barton

Discussing if its worth moving Py/C functions from METH_VARARGS to METHO when they only recieve 1 argument on the PyGame mailing list. Crossposting here since some of you might be interested in the results. tested different ways to evaluate args to see how much speed difference there was * 10,000,000 tests, python 2.6 on 32bit arch linux * included a pass and NOARGS metrhod to see the difference in overhead of the loop and parsing an arg compared to running a method with no args. ---- output pass 1.85659885406 METH_NOARGS 3.24079704285 METH_O 3.66321516037 METH_VARARGS 6.09881997108 METH_KEYWORDS 6.037307024 METH_KEYWORDS (as keyword) 10.9263861179 ------- Python Script, (used blender module for testing) import time from Blender.sys import test_METHO, test_METH_VARARGS, test_METH_KEYWORDS, test_METH_NOARGS RUN = 10000000 t = time.time() for i in xrange(RUN): pass print 'pass', time.time()-t t = time.time() for i in xrange(RUN): test_METH_NOARGS() print 'METH_NOARGS', time.time()-t t = time.time() for i in xrange(RUN): test_METHO(1) print 'METH_O', time.time()-t t = time.time() for i in xrange(RUN): test_METH_VARARGS(1) print 'METH_VARARGS', time.time()-t t = time.time() for i in xrange(RUN): test_METH_KEYWORDS(1) print 'METH_KEYWORDS', time.time()-t t = time.time() for i in xrange(RUN): test_METH_KEYWORDS(val=1) print 'METH_KEYWORDS (as keyword)', time.time()-t ------------------- C functions static PyObject *test_METHO( PyObject * self, PyObject * value ) { int val = (int)PyInt_AsLong(value); if( val==-1 && PyErr_Occurred() ) { PyErr_SetString(PyExc_AttributeError, "not an int"); return NULL; } Py_RETURN_NONE; } static PyObject *test_METH_VARARGS( PyObject * self, PyObject * args ) { int val; if( !PyArg_ParseTuple( args, "i", &val ) ) return NULL; Py_RETURN_NONE; } static PyObject *test_METH_KEYWORDS( PyObject * self, PyObject * args, PyObject *kwd) { int val; static char *kwlist[] = {"val", NULL}; if( !PyArg_ParseTupleAndKeywords(args, kwd, "i", kwlist, &val) ) return NULL; Py_RETURN_NONE; } static PyObject *test_METH_NOARGS( PyObject * self, PyObject * args ) { Py_RETURN_NONE; } struct PyMethodDef M_sys_methods[] = { {"test_METHO", test_METHO, METH_O, ""}, {"test_METH_KEYWORDS", test_METH_KEYWORDS, METH_KEYWORDS, ""}, {"test_METH_NOARGS", test_METH_NOARGS, METH_NOARGS, ""}, {"test_METH_VARARGS", test_METH_VARARGS, METH_VARARGS, ""}, {NULL, NULL, 0, NULL} }; -- - Campbell

13 years, 1 month

Re: [capi-sig] Exceptions with additional instance variables

by chojrak11＠gmail.com

2008/12/22 Guilherme Polo <ggpolo(a)gmail.com>: > On Mon, Dec 22, 2008 at 10:06 AM, <chojrak11(a)gmail.com> wrote: > > #include "Python.h" > > static PyObject *MyErr; > > static PyMethodDef module_methods[] = { > {"raise_test1", (PyCFunction)raise_test1, METH_NOARGS, NULL}, > {"raise_test2", (PyCFunction)raise_test2, METH_NOARGS, NULL}, > {"raise_test3", (PyCFunction)raise_test3, METH_NOARGS, NULL}, > {NULL}, > }; > > PyMODINIT_FUNC > initfancy_exc(void) > { > PyObject *m; > > m = Py_InitModule("fancy_exc", module_methods); > if (m == NULL) > return; > > MyErr = PyErr_NewException("fancy_exc.err", NULL, NULL); > > Py_INCREF(MyErr); > if (PyModule_AddObject(m, "err", MyErr) < 0) > return; > } > > static PyObject * > raise_test1(PyObject *self) > { > PyObject_SetAttrString(MyErr, "code", PyInt_FromLong(42)); > PyObject_SetAttrString(MyErr, "category", PyString_FromString("nice one")); > PyErr_SetString(MyErr, "All is good, I hope"); > return NULL; > } > > static PyObject * > raise_test2(PyObject *self) > { > > PyObject *t = PyTuple_New(3); > PyTuple_SetItem(t, 0, PyString_FromString("error message")); > PyTuple_SetItem(t, 1, PyInt_FromLong(10)); > PyTuple_SetItem(t, 2, PyString_FromString("category name here")); > PyErr_SetObject(MyErr, t); > Py_DECREF(t); > return NULL; > } > > In this second form you check for the args attribute of the exception. static PyObject * raise_test3(PyObject *self) { PyObject *d = PyDict_New(); PyDict_SetItemString(d, "category", PyInt_FromLong(111)); PyDict_SetItemString(d, "message", PyString_FromString("error message")); PyErr_SetObject(MyErr, d); Py_DECREF(d); return NULL; } (Small changes in the above code to be able to call more variants of raise_test methods simultaneously.) Yes! I finally understood this (I think...) So to explain things for people like me: 1) PyErr_NewException creates *the class* in the module, it's a simple method of creating exception classes, but classes created that way are limited in features (i.e. cannot be manipulated from the module in all ways a 'full' type can). Third argument to PyErr_NewException can be NULL, in which case API will create an empty dictionary. After creating the class you need to add it to the module with PyModule_AddObject. Side note: If you want to specify a help for the class, you do PyObject_SetAttrString on the class with the key '__doc__'. 2) there's no instantiation anywhere: a. PyErr_SetString and PyErr_SetObject set the exception *class* (exception type) and exception data -- see http://docs.python.org/c-api/exceptions.html which notes that exceptions are similar in concept to the global 'errno' variable, so you just set what type of last error was and what error message (or other data) you want to associate with it b. the "code" and "category" variables from raise_test1() in the above example inserted with PyObject_SetAttrString() are *class* variables, not instance variables: try: fancy_exc.raise_test1() except fancy_exc.err, e: print e.code, fancy_exc.err.code print fancy_exc.err.code it prints: 42 42 42 c. the data is still present in the fancy_exc.err class after exception handling is finished, which is ok for now but may be problematic in case of multithreaded usage patterns (however I probably don't understand how multithreading in Python works) 3) alternative to the above is to pass all required data to the exception with PyErr_SetObject - you can prepare a dictionary or a tuple earlier, which will be accessible with 'args' member: try: fancy_exc.raise_test2() except fancy_exc.err, e: print e.args[0] If it's dictionary, the syntax is a bit weird because e.args is always a tuple: try: fancy_exc.raise_test3() except fancy_exc.err, e: print e.args[0]['category'] The 'args' values are unavailable outside of 'except' clause, however you can still use the 'e' variable which retains the values. So it's an instance variable. 4) creating the exception class using a new type in C (PyTypeObject structure) would give the most robust solution because every nuance of the class can be manipulated, but it's not worth the trouble now. I can switch to it transparently at a later time. Transparently means that nothing will need to be updated in Python solutions written by the module users. 5) most of the projects I've inspected with Google Code Search use the PyErr_NewException approach. 6) there's the option of using Cython which simplifies creating extensions and hides many unnecessary internals. Many thanks Guilherme and Stefan for your help and for the patience. Kind regards, Chojrak

13 years, 1 month

Re: [capi-sig] [Fwd: Re: Exceptions with additional instance variables]

by chojrak11＠gmail.com

2008/12/22 Stefan Behnel <python_capi(a)behnel.de>: > The quick way to fix your problem might be to write the exception code in > Cython like thís: > > class MyException(Exception): > def __init__(self, errorcode, message): > super(MyException, self).__init__(self, message) > self.errorcode = errorcode > > and use that instead. You can also make it an exception class if you like, > by declaring the Exception class as an external type (requires Py2.5+). In > this case, Cython can generate a header file for you that declares your > new extension type. > > http://cython.org/ Hello Stefan, thanks for the tip, I'll surely look at Cython and how it can help ;-) Thanks for the pointer and for the example! Regards, Chojrak

13 years, 1 month

[Fwd: Re: Exceptions with additional instance variables]

by Stefan Behnel

[... and to the list ...] chojrak11(a)gmail.com wrote: > Ok so I did the following. In init function (forget refcounting and > error checking for a moment ;-) > > PyObject *dict = PyDict_New(); > PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0)); > static PyObject *myexception = > PyErr_NewException("module.MyException", NULL, dict); > PyModule_AddObject(module, "MyException", myexception); > > It worked more or less as expected, the help shown: > > | ---------------------------------------------------------------------- > | Data and other attributes defined here: > | > | errorcode = 0 > | > | ---------------------------------------------------------------------- > > Then I did the following when raising the exception: > > PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111)); > PyErr_SetString(myexception, "Bad thing happened"); > return NULL; > > and the test code was: > try: > do_bad_thing(); > except MyException, data: > > and you surely already guessed it -- data.errorcode was 0.... Not only > that, module.MyException.errorcode was also 0... > > What I'm doing wrong? I certainly don't get the idea of exceptions in > Python, especially what is being raised - a class or an instance? If > the latter - how's the class instantiated? If not - what about values > in different threads? The docs are so vague about that... The quick way to fix your problem might be to write the exception code in Cython like thís: class MyException(Exception): def __init__(self, errorcode, message): super(MyException, self).__init__(self, message) self.errorcode = errorcode and use that instead. You can also make it an exception class if you like, by declaring the Exception class as an external type (requires Py2.5+). In this case, Cython can generate a header file for you that declares your new extension type. http://cython.org/ Stefan

13 years, 1 month

Re: [capi-sig] Exceptions with additional instance variables

by chojrak11＠gmail.com

On Mon, Dec 22, 2008 at 03:29, Guilherme Polo <ggpolo(a)gmail.com> wrote: > On Sun, Dec 21, 2008 at 11:02 PM, <chojrak11(a)gmail.com> wrote: >> Hello, >> >> I'm trying to implement custom exception that have to carry some >> useful info by means of instance members, to be used like: >> >> try: >> // some code >> except MyException, data: >> // use data.errorcode, data.errorcategory, data.errorlevel, >> data.errormessage and some others >> >> The question is - how to implement the instance variables with >> PyErr_NewException? > > Using PyErr_NewException is fine. You must understand that an > exception is a class, and thus PyErr_NewException creates one for you > and returns it. > Just like you would do with a class that has __dict__, set some > attributes to what you want. That is, use PyObject_SetAttrString or > something more appropriated for you. Ok so I did the following. In init function (forget refcounting and error checking for a moment ;-) PyObject *dict = PyDict_New(); PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0)); static PyObject *myexception = PyErr_NewException("module.MyException", NULL, dict); PyModule_AddObject(module, "MyException", myexception); It worked more or less as expected, the help shown: | ---------------------------------------------------------------------- | Data and other attributes defined here: | | errorcode = 0 | | ---------------------------------------------------------------------- Then I did the following when raising the exception: PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111)); PyErr_SetString(myexception, "Bad thing happened"); return NULL; and the test code was: try: do_bad_thing(); except MyException, data: and you surely already guessed it -- data.errorcode was 0.... Not only that, module.MyException.errorcode was also 0... What I'm doing wrong? I certainly don't get the idea of exceptions in Python, especially what is being raised - a class or an instance? If the latter - how's the class instantiated? If not - what about values in different threads? The docs are so vague about that... Thanks again in advance, Chojrak

13 years, 1 month

Exceptions with additional instance variables

by chojrak11＠gmail.com

Hello, I'm trying to implement custom exception that have to carry some useful info by means of instance members, to be used like: try: // some code except MyException, data: // use data.errorcode, data.errorcategory, data.errorlevel, data.errormessage and some others The question is - how to implement the instance variables with PyErr_NewException? Or should I use another approach? And how to set values of these variables at the moment of exception? Should I use PyErr_SetString or PyErr_SetObject? I'm lost. Thanks in advance. Kind regards.

13 years, 1 month

ANN: PyBindGen 0.10 released

by Gustavo Carneiro

PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write. It can be downloaded from: http://code.google.com/p/pybindgen/ Bug reports should be filed here: https://bugs.launchpad.net/pybindgen === pybindgen 0.10 === - New null_ok, default_value options for pointer-to-class parameters; - Thread safety fixes; - Map C++ operator() into Python's tp_call (__call__); - Initial support for std containers (except mapping containers); - Generate __copy__ methods for classes with copy constructor; - Add 'wrapper registry' optional feature (off by default): allows C++ instances returned by pointer to be consistently wrapped always by the same Python wrapper, so that the 'is' operator can be used for identity testing; - New C type expression parser, is_const not longer needed for parameter types, just put const in the type string as appropriate; - Sort the declarations returned by (py)gccxml. If generating python script files from gccxml scanning, now the output file becomes more stable and less prone to move declarations up and down with different people scanning and the output file is under version control; - Map binary comparison operators (< <= == >= >) from C++ to Python; - Map some binary numeric operators (+ - * /) from C++ to Python; - Allow installation of pybindgen even if no C/C++ compiler is detected; - Fix compatibility with older Python versions (tested with 2.3, 2.4, and 2.5) -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert

13 years, 1 month

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

capi-sig December 2008