Recently I wanted to deprecate some functions in our the C api, and
found that warnings give weired output when running through through
our C api.
A simple test is to do this...
print range(1.5)
When I run this from python I get good output..
test.py:1: DeprecationWarning: integer argument expected, got float
print range(1.5)
But when running from our C api I get
/home/ideasman42/blender-svn/blender/blender.bin:1:
DeprecationWarning: integer argument expected, got float
ELF�p40�"4()&44�4�44�4�����F��F��F�K �K ğ<@G�K �K xxHH�H�
P�td�2?��C ��C L,L,Q�td/lib/ld-linux.so.2GNU����8�1LQ
�U�N�.yH {��d
I thought this was todo with running a compiled string so I tried
PyRun_String() on the text buffer instead of PyEval_EvalCode() but the
same problem.
Id be interested to know if anyones embedded python gives proper warnings.
On Linux testing with python 2.6 here.
--
- Campbell
Parsing tuples into C arrays is quite painful
with current C API. Below is is an example
when you don't know the tuple size before hand:
....
//ref: http://docs.python.org/c-api/arg.html
PyObject* pyObj;
if(!PyArg_ParseTuple(args,"O",&pyObj)
|| !pyTuple_Check(pyObj))
return NULL;
tupleSize = PyTuple_Size(pyObj);
array = (double*) malloc(sizeof(double)*tupleSize);
for(i=0; i < tupleSize; i++ ) {
tupleItem = PyTuple_GetItem(pyObj, i);
if( PyFloat_Check(tupleItem) ) {
array[i] = PyFloat_AsDouble(tupleItem);
}else{
printf("Error: tuple contains a non-float value");
exit(1);
}
}
....
I'd like to propose a syntax to facilitate that kind
of tasks. There is already syntax for parsing strings:
{
const char * s; int i;
PyArg_ParseTyple(args, "s#", &s, &i);
}
Consider the string as a sequence of char's.
Then note that in python, a tuple of int's,
is also a sequence of int's. This
perspective provides a natural way to
extend the the use of "#" symbol.
First, to denote a tuple as a sequence
of a given type of objects, let's use
"(<type>:)". For example, to denote a tuple
as a sequence of integers: "(i:)".
The ":" is chosen to denote possible
repetition of the whole pattern inside
the parentheses. As for why choose ":",
here is a bad story, but a good example:
Once upone a time there was a temple on
a hill, and there lived an old and a
young monk. One day the old monk told the
the young a story:
Once upone a time there was a temple on
a hill, and there lived an old and a
young monk. One day the old monk told the
the young a story:
...
OK, enough...but you got the idea :)
Now consider 3 different scenarios of parsing
tuples into arrays.
(1) Unkown size. Here are some examples:
{
//parsing a tuple and its size
int i, *t;
PyArg_ParseTuple(args, "(i:)#", &t, &i);
}
{
//parsing a matrix
int i,j,**t;
PyArg_ParseTuple(args, "((i:):)##", &t, &i, &j);
for(a=0; a<i; a++){
for(b=0; b<j; b++)
printf("%d, ", t[a][b]);
printf("\n");
}
}
Here is an informal explanation of "((i:):)##":
the '#' can be seen as a peeling operator, which
peels away the outer most ':'-parentheses pair "(.:)"
that denote a "free sequence" (i.e. sequence of
unknown size) with some pattern ".".
For types like 's' or its similar types, pretend
as if there is an invisible pair of parentheses
around it. For example, "(s:)##" matches a tuple
of strings of some fixed length. Here the first
'#' peels away the parentheses, leaving only "s#",
which is equivalent to "(<char>:)#" in concept,
where "<char>" denote the character type.
As the second '#' is outside the tuple,
it is shared by all strings inside, so they
must have the same length.
If there is nothing to peel for a '#', it is
considered a dangling '#', an error.
Now consider some more interesting examples:
{
int i, *t, *v;
PyArg_ParseTuple(args, "(ii:)#", &t, &v, &i);
//args=(1,2,3,4,5,6)
//t={1,3,5}
//v={2,4,6}
//i=3
}
In that example, a pattern is provided inside the sequence "(ii:)",
which can repeat itself multiple times. The repeated times is
extracted by the '#' symbol. Below is another example:
{
int i,j, *t, **v;
PyArg_ParseTuple(args, "(i(i:):)##", &t, &v, &i, &j);
//args=(1,(3,),5,(7,))
//t={1,5}
//v={{3},{7}}
//i=2, j=1
}
And another one:
{
int i,j, **t, **v;
PyArg_ParseTuple(args, "((i:)(i:):)##", &t, &v, &i, &j);
//args=((1,),(3,),(5,),(7,))
//t={{1},{5}}
//v={{3},{7}}
//i=2, j=1
}
Note the second '#' is applied to both the free sequences inside the
outer tuple. This indicates that all the inside tuples must be of
the same length. What if they have their own common lengths? We
provide a separate matching '#' for each inside tuple:
{
int i,j,k, **t, **v;
PyArg_ParseTuple(args, "((i:)(i:):)#(##)", &t, &v, &i, &j,&k);
//args=((1,),(3,4),(5,),(7,8))
//t={{1},{5}}
//v={{3,4},{7,8}}
//i=2, j=1, k=2
}
The syntax can also deal with more complicated situations,
such as an irregular 2-dimensional array, where each row
has a different row length:
{
int i, *j, **t;
PyArg_ParseTuple(args, "((i:)#:)#", &t, &j, &i);
for(a=0; a<i; a++){
for(b=0; b<j[i]; b++)
printf("%d, ", t[a][b]);
printf("\n");
}
//args=((1,2), (1,2,3))
//t={{1,2},{1,2,3}}
//i=2, j={2,3}
}
Here is a mixed example: three dimensional
array 't' has fixed length on the third
dimention 'k', but has variable length on the
second dimension 'j':
{
int i, *j, k, ***t;
PyArg_ParseTuple(args, "(((i:):)#:)##", &t, &j, &i, &k);
//args=(((1,2),(1,2)),((1,2),))
//t={{{1,2},{1,2}},{{1,2}}}
//i=2, k=2
//j={2,1}
}
This syntax can be used for parsing a tuple of strings:
{
int i, *j;
const char ** s;
PyArg_ParseTuple(args, "(s#:)#", &s, &j, &i);
//args=("ab", "cde")
//s={"ab", "cde"}
//j={2, 3}
//i=2
}
(2) If you know the size of a tuple before hand,
use this syntax: "(<type>integer)", meaning there
are exactly 'integer' number of <type> objects in
the tuple.
{
int *t;
//
PyArg_ParseTuple(args, "(i5)", &t);
//args = (1,2,3,4,5)
//t = {1,2,3,4,5}
}
A complicated case:
{
int **t;
PyArg_ParseTuple(args, "((i5)2)", &t);
//args=((1,2,3,4,5),(5,6,7,8,9))
//t={{1,2,3,4,5},{5,6,7,8,9}}
}
Another complicated case:
{
int **t, **v, i;
PyArg_ParseTuple(args, "(i3i2:)#", &t,&v,&i);
//args=(1,2,3,4,5,6,7,8,9,10)
//t={{1,2,3},{6,7,8}}
//v={{4,5},{9,10}}
//i=2
}
One more thing: the difference between
"(ii)" and "(i2)" is that the former
yields two single integers, while the
latter gives an array of length two.
(3) Concatenating arrays while parsing.
Sometimes you would like to have a matrix
parsed into a single array. Let's use a "(+"
to start a tuple, such that the arrays
parsed from the elements in that tuple
should be concatenated into a single array.
{
int i,j, *t;
PyArg_ParseTuple(args, "(+(i))##", &t, &i, &j);
//args=((1,2,3),(4,5,6))
//t={1,2,3,4,5,6}
//i=2, j=3
}
Another example:
{
int i, *t;
PyArg_ParseTuple(args, "(+(i2))#", &t, &i);
}
Still another example, with strings:
{
int i, *j;
const char * t; //string
PyArg_ParseTuple(args, "(+(+s#2))#", &t, &j, &i);
//args=(("ab","c"),("d","efg"))
//t="abcdefg"
//i=2
////notice the size arrays are contatenated similarly
//j={2,1,1,3}
const char ** sa;
PyArg_ParseTuple(args, "(+(s2))", &sa);
//sa = {"ab","c","d","efg"}
PyArg_ParseTuple(args, "((+s2))", &sa);
//sa = { "abc", "defg" }
}
And finally, an example with multi-item pattern:
{
int *t, *v, i;
PyArg_ParseTuple(args, "(+i1i2:)#", &t, &v, &i);
//args=(1,2,3,4,5,6)
//t={1,4}
//v={2,3,5,6}
//i=2
}
Epilogue:
This syntax can be applied to lists:
just use "[.:]" instead of "(.:)".
It can also be used for building tuples
or lists out of arrays.
That's it. Comments?
-Yingjie
Hello,
I have a Python module here that takes a bunch of C++ classes, and turns
them into Python classes. When built against and loaded into Python
2.4, everything is fine. But when built against and loaded into Python
2.5, I get an assert violation from Python's gc module. Here is the
code that is failing:
klass = PyClass_New(bases, classDict, className);
if (klass && methods) {
/* add methods to class */
for (def = methods; def->ml_name != NULL; def++) {
printf( "IlmPyClass: %d, def = %s\n", __LINE__, def->ml_name);
PyObject *func = IlmPyClass_NewFunction(def);
if (!func) {
Py_XDECREF(klass);
return NULL;
}
printf( "We get here\n" );
func = PyMethod_New(func, NULL, klass); //this line fails
printf( "We don't get here\n" );
# .......
}
}
The output of 'python2.5 -c "import mymod"' is:
"""
[.... snip a bunch of "we get here, we don't get here" etc. as things
fail to fail...]
IlmPyClass: 197, def = __init__
We get here
python2: Modules/gcmodule.c:276: visit_decref: Assertion `gc->gc.gc_refs
!= 0' failed.
Abort
"""
The obvious things, such as Py_INCREFing klass or func, do not work.
What's extra strange, in addition to this code working fine in an
earlier python version, is that this code works fine for most of the
classes that are instantiated.
What I really want to do, though, is see what exactly is being decref'd.
Does anyone have any tips for doing this? I've already tried using a
debug build of Python; it doesn't seem to provide that type of ref
tracing. Thanks in advance for any tips or insights.
--
Joe Ardent
