Discussing if its worth moving Py/C functions from METH_VARARGS to
METHO when they only recieve 1 argument on the PyGame mailing list.
Crossposting here since some of you might be interested in the results.
tested different ways to evaluate args to see how much speed
difference there was
* 10,000,000 tests, python 2.6 on 32bit arch linux
* included a pass and NOARGS metrhod to see the difference in overhead
of the loop and parsing an arg compared to running a method with no
args.
---- output
pass 1.85659885406
METH_NOARGS 3.24079704285
METH_O 3.66321516037
METH_VARARGS 6.09881997108
METH_KEYWORDS 6.037307024
METH_KEYWORDS (as keyword) 10.9263861179
------- Python Script, (used blender module for testing)
import time
from Blender.sys import test_METHO, test_METH_VARARGS,
test_METH_KEYWORDS, test_METH_NOARGS
RUN = 10000000
t = time.time()
for i in xrange(RUN): pass
print 'pass', time.time()-t
t = time.time()
for i in xrange(RUN): test_METH_NOARGS()
print 'METH_NOARGS', time.time()-t
t = time.time()
for i in xrange(RUN): test_METHO(1)
print 'METH_O', time.time()-t
t = time.time()
for i in xrange(RUN): test_METH_VARARGS(1)
print 'METH_VARARGS', time.time()-t
t = time.time()
for i in xrange(RUN): test_METH_KEYWORDS(1)
print 'METH_KEYWORDS', time.time()-t
t = time.time()
for i in xrange(RUN): test_METH_KEYWORDS(val=1)
print 'METH_KEYWORDS (as keyword)', time.time()-t
------------------- C functions
static PyObject *test_METHO( PyObject * self, PyObject * value )
{
int val = (int)PyInt_AsLong(value);
if( val==-1 && PyErr_Occurred() ) {
PyErr_SetString(PyExc_AttributeError, "not an int");
return NULL;
}
Py_RETURN_NONE;
}
static PyObject *test_METH_VARARGS( PyObject * self, PyObject * args )
{
int val;
if( !PyArg_ParseTuple( args, "i", &val ) )
return NULL;
Py_RETURN_NONE;
}
static PyObject *test_METH_KEYWORDS( PyObject * self, PyObject * args,
PyObject *kwd)
{
int val;
static char *kwlist[] = {"val", NULL};
if( !PyArg_ParseTupleAndKeywords(args, kwd, "i", kwlist, &val) )
return NULL;
Py_RETURN_NONE;
}
static PyObject *test_METH_NOARGS( PyObject * self, PyObject * args )
{
Py_RETURN_NONE;
}
struct PyMethodDef M_sys_methods[] = {
{"test_METHO", test_METHO, METH_O, ""},
{"test_METH_KEYWORDS", test_METH_KEYWORDS, METH_KEYWORDS, ""},
{"test_METH_NOARGS", test_METH_NOARGS, METH_NOARGS, ""},
{"test_METH_VARARGS", test_METH_VARARGS, METH_VARARGS, ""},
{NULL, NULL, 0, NULL}
};
--
- Campbell
2008/12/22 Guilherme Polo <ggpolo(a)gmail.com>:
> On Mon, Dec 22, 2008 at 10:06 AM, <chojrak11(a)gmail.com> wrote:
>
> #include "Python.h"
>
> static PyObject *MyErr;
>
> static PyMethodDef module_methods[] = {
> {"raise_test1", (PyCFunction)raise_test1, METH_NOARGS, NULL},
> {"raise_test2", (PyCFunction)raise_test2, METH_NOARGS, NULL},
> {"raise_test3", (PyCFunction)raise_test3, METH_NOARGS, NULL},
> {NULL},
> };
>
> PyMODINIT_FUNC
> initfancy_exc(void)
> {
> PyObject *m;
>
> m = Py_InitModule("fancy_exc", module_methods);
> if (m == NULL)
> return;
>
> MyErr = PyErr_NewException("fancy_exc.err", NULL, NULL);
>
> Py_INCREF(MyErr);
> if (PyModule_AddObject(m, "err", MyErr) < 0)
> return;
> }
>
> static PyObject *
> raise_test1(PyObject *self)
> {
> PyObject_SetAttrString(MyErr, "code", PyInt_FromLong(42));
> PyObject_SetAttrString(MyErr, "category", PyString_FromString("nice one"));
> PyErr_SetString(MyErr, "All is good, I hope");
> return NULL;
> }
>
> static PyObject *
> raise_test2(PyObject *self)
> {
>
> PyObject *t = PyTuple_New(3);
> PyTuple_SetItem(t, 0, PyString_FromString("error message"));
> PyTuple_SetItem(t, 1, PyInt_FromLong(10));
> PyTuple_SetItem(t, 2, PyString_FromString("category name here"));
> PyErr_SetObject(MyErr, t);
> Py_DECREF(t);
> return NULL;
> }
>
> In this second form you check for the args attribute of the exception.
static PyObject *
raise_test3(PyObject *self) {
PyObject *d = PyDict_New();
PyDict_SetItemString(d, "category", PyInt_FromLong(111));
PyDict_SetItemString(d, "message", PyString_FromString("error
message"));
PyErr_SetObject(MyErr, d);
Py_DECREF(d);
return NULL;
}
(Small changes in the above code to be able to call more variants of
raise_test methods simultaneously.)
Yes! I finally understood this (I think...) So to explain things for
people like me:
1) PyErr_NewException creates *the class* in the module, it's a simple
method of creating exception classes, but classes created that way are
limited in features (i.e. cannot be manipulated from the module in all
ways a 'full' type can). Third argument to PyErr_NewException can be
NULL, in which case API will create an empty dictionary. After
creating the class you need to add it to the module with
PyModule_AddObject. Side note: If you want to specify a help for the
class, you do PyObject_SetAttrString on the class with the key
'__doc__'.
2) there's no instantiation anywhere:
a. PyErr_SetString and PyErr_SetObject set the exception *class*
(exception type) and exception data -- see
http://docs.python.org/c-api/exceptions.html which notes that
exceptions are similar in concept to the global 'errno' variable, so
you just set what type of last error was and what error message (or
other data) you want to associate with it
b. the "code" and "category" variables from raise_test1() in the
above example inserted with PyObject_SetAttrString() are *class*
variables, not instance variables:
try:
fancy_exc.raise_test1()
except fancy_exc.err, e:
print e.code, fancy_exc.err.code
print fancy_exc.err.code
it prints:
42 42
42
c. the data is still present in the fancy_exc.err class after
exception handling is finished, which is ok for now but may be
problematic in case of multithreaded usage patterns (however I
probably don't understand how multithreading in Python works)
3) alternative to the above is to pass all required data to the
exception with PyErr_SetObject - you can prepare a dictionary or a
tuple earlier, which will be accessible with 'args' member:
try:
fancy_exc.raise_test2()
except fancy_exc.err, e:
print e.args[0]
If it's dictionary, the syntax is a bit weird because e.args is always a tuple:
try:
fancy_exc.raise_test3()
except fancy_exc.err, e:
print e.args[0]['category']
The 'args' values are unavailable outside of 'except' clause, however
you can still use the 'e' variable which retains the values. So it's
an instance variable.
4) creating the exception class using a new type in C (PyTypeObject
structure) would give the most robust solution because every nuance of
the class can be manipulated, but it's not worth the trouble now. I
can switch to it transparently at a later time. Transparently means
that nothing will need to be updated in Python solutions written by
the module users.
5) most of the projects I've inspected with Google Code Search use the
PyErr_NewException approach.
6) there's the option of using Cython which simplifies creating
extensions and hides many unnecessary internals.
Many thanks Guilherme and Stefan for your help and for the patience.
Kind regards,
Chojrak
2008/12/22 Stefan Behnel <python_capi(a)behnel.de>:
> The quick way to fix your problem might be to write the exception code in
> Cython like thís:
>
> class MyException(Exception):
> def __init__(self, errorcode, message):
> super(MyException, self).__init__(self, message)
> self.errorcode = errorcode
>
> and use that instead. You can also make it an exception class if you like,
> by declaring the Exception class as an external type (requires Py2.5+). In
> this case, Cython can generate a header file for you that declares your
> new extension type.
>
> http://cython.org/
Hello Stefan,
thanks for the tip, I'll surely look at Cython and how it can help ;-)
Thanks for the pointer and for the example!
Regards,
Chojrak
[... and to the list ...]
chojrak11(a)gmail.com wrote:
> Ok so I did the following. In init function (forget refcounting and
> error checking for a moment ;-)
>
> PyObject *dict = PyDict_New();
> PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0));
> static PyObject *myexception =
> PyErr_NewException("module.MyException", NULL, dict);
> PyModule_AddObject(module, "MyException", myexception);
>
> It worked more or less as expected, the help shown:
>
> | ----------------------------------------------------------------------
> | Data and other attributes defined here:
> |
> | errorcode = 0
> |
> | ----------------------------------------------------------------------
>
> Then I did the following when raising the exception:
>
> PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111));
> PyErr_SetString(myexception, "Bad thing happened");
> return NULL;
>
> and the test code was:
> try:
> do_bad_thing();
> except MyException, data:
>
> and you surely already guessed it -- data.errorcode was 0.... Not only
> that, module.MyException.errorcode was also 0...
>
> What I'm doing wrong? I certainly don't get the idea of exceptions in
> Python, especially what is being raised - a class or an instance? If
> the latter - how's the class instantiated? If not - what about values
> in different threads? The docs are so vague about that...
The quick way to fix your problem might be to write the exception code in
Cython like thís:
class MyException(Exception):
def __init__(self, errorcode, message):
super(MyException, self).__init__(self, message)
self.errorcode = errorcode
and use that instead. You can also make it an exception class if you like,
by declaring the Exception class as an external type (requires Py2.5+). In
this case, Cython can generate a header file for you that declares your
new extension type.
http://cython.org/
Stefan
On Mon, Dec 22, 2008 at 03:29, Guilherme Polo <ggpolo(a)gmail.com> wrote:
> On Sun, Dec 21, 2008 at 11:02 PM, <chojrak11(a)gmail.com> wrote:
>> Hello,
>>
>> I'm trying to implement custom exception that have to carry some
>> useful info by means of instance members, to be used like:
>>
>> try:
>> // some code
>> except MyException, data:
>> // use data.errorcode, data.errorcategory, data.errorlevel,
>> data.errormessage and some others
>>
>> The question is - how to implement the instance variables with
>> PyErr_NewException?
>
> Using PyErr_NewException is fine. You must understand that an
> exception is a class, and thus PyErr_NewException creates one for you
> and returns it.
> Just like you would do with a class that has __dict__, set some
> attributes to what you want. That is, use PyObject_SetAttrString or
> something more appropriated for you.
Ok so I did the following. In init function (forget refcounting and
error checking for a moment ;-)
PyObject *dict = PyDict_New();
PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0));
static PyObject *myexception =
PyErr_NewException("module.MyException", NULL, dict);
PyModule_AddObject(module, "MyException", myexception);
It worked more or less as expected, the help shown:
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| errorcode = 0
|
| ----------------------------------------------------------------------
Then I did the following when raising the exception:
PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111));
PyErr_SetString(myexception, "Bad thing happened");
return NULL;
and the test code was:
try:
do_bad_thing();
except MyException, data:
and you surely already guessed it -- data.errorcode was 0.... Not only
that, module.MyException.errorcode was also 0...
What I'm doing wrong? I certainly don't get the idea of exceptions in
Python, especially what is being raised - a class or an instance? If
the latter - how's the class instantiated? If not - what about values
in different threads? The docs are so vague about that...
Thanks again in advance,
Chojrak
Hello,
I'm trying to implement custom exception that have to carry some
useful info by means of instance members, to be used like:
try:
// some code
except MyException, data:
// use data.errorcode, data.errorcategory, data.errorlevel,
data.errormessage and some others
The question is - how to implement the instance variables with
PyErr_NewException? Or should I use another approach? And how to set
values of these variables at the moment of exception? Should I use
PyErr_SetString or PyErr_SetObject?
I'm lost.
Thanks in advance.
Kind regards.
PyBindGen is a Python module that is geared to generating C/C++ code that
binds a C/C++ library for Python. It does so without extensive use of either
C++ templates or C pre-processor macros. It has modular handling of C/C++
types, and can be easily extended with Python plugins. The generated code is
almost as clean as what a human programmer would write.
It can be downloaded from:
http://code.google.com/p/pybindgen/
Bug reports should be filed here:
https://bugs.launchpad.net/pybindgen
=== pybindgen 0.10 ===
- New null_ok, default_value options for pointer-to-class parameters;
- Thread safety fixes;
- Map C++ operator() into Python's tp_call (__call__);
- Initial support for std containers (except mapping containers);
- Generate __copy__ methods for classes with copy constructor;
- Add 'wrapper registry' optional feature (off by default): allows
C++ instances returned by pointer to be consistently wrapped
always by the same Python wrapper, so that the 'is' operator can
be used for identity testing;
- New C type expression parser, is_const not longer needed for
parameter types, just put const in the type string as
appropriate;
- Sort the declarations returned by (py)gccxml. If generating
python script files from gccxml scanning, now the output file
becomes more stable and less prone to move declarations up and
down with different people scanning and the output file is under
version control;
- Map binary comparison operators (< <= == >= >) from C++ to Python;
- Map some binary numeric operators (+ - * /) from C++ to Python;
- Allow installation of pybindgen even if no C/C++ compiler is detected;
- Fix compatibility with older Python versions (tested with 2.3, 2.4,
and 2.5)
--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
