Possible C API problem?
Hello, I was asking about a problem I was having over on the C++-python list, and they suggested I report it here as a possible Python problem. I was getting bus errors with a C module I was linking to, so factored it down too a very small example that reproduced the problem. Here it is: #include <Python.h> static double gfSumChiSquare = 123.0; static PyObject * getSumChiSquare(PyObject *self, PyObject *args){ return Py_BuildValue("d", gfSumChiSquare); } static PyMethodDef SimMethods[] = { {"getSumChiSquare", getSumChiSquare, METH_NOARGS, "Return fSumChiSquare"}, {NULL, NULL, 0, NULL} /* Sentinel */ }; PyMODINIT_FUNC inittestfloat(void) { (void) Py_InitModule("testfloat", SimMethods); } That caused a bus error 100% of the time when I simply imported the module into Python and called getSumChiSquare(), i.e.:
import testfloat testfloat.getSumChiSquare()
However, the problem seems to go away if I use METH_VARARGS, and parse the non-existent args with PyArg_ParseTuple: #include <Python.h> static double gfSumChiSquare = 123.0; static PyObject * getSumChiSquare(PyObject *self, PyObject *args){ if (!PyArg_ParseTuple(args, "", NULL)) return NULL; return Py_BuildValue("d", gfSumChiSquare); } static PyMethodDef SimMethods[] = { {"getSumChiSquare", getSumChiSquare, METH_VARARGS, "Return fSumChiSquare"}, {NULL, NULL, 0, NULL} /* Sentinel */ }; PyMODINIT_FUNC inittestfloat(void) { (void) Py_InitModule("testfloat", SimMethods); } This approach seems to work reliably -- at least variations I've tried haven't caused a bus error. But I haven't been able to discern an explanation from the docs as to why this would be better. The docs say that both METH_VARARGS and METH_NOARGS expect a PyCFunction. So if I am calling the function with no arguments, why can't I use METH_NOARGS and skip the call to PyArg_ParseTuple? Could it be that this is a python bug? Or am I doing something wrong? Note: this is using Python 2.3 on OS X: Python 2.3 (#1, Sep 13 2003, 00:49:11) Thanks in advance for any help or insight you can give, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson@goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net
Gary Robinson <grobinson@goombah.com> writes:
That caused a bus error 100% of the time when I simply imported the module into Python and called getSumChiSquare(), i.e.:
import testfloat testfloat.getSumChiSquare()
It doesn't for me (CVS HEAD, OS X Panther).
Could it be that this is a python bug? Or am I doing something wrong?
Note: this is using Python 2.3 on OS X:
Python 2.3 (#1, Sep 13 2003, 00:49:11)
Thanks in advance for any help or insight you can give,
Have you, you know, tried to debug the situation yourself? If you have gcc installed, you probably have gdb installed too... Cheers, mwh -- You can lead an idiot to knowledge but you cannot make him think. You can, however, rectally insert the information, printed on stone tablets, using a sharpened poker. -- Nicolai -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
Michael Hudson wrote:
Gary Robinson <grobinson@goombah.com> writes:
... bus error 100% of the time ...:
We've boiled it down pretty far, and I've sent him off to the mac-python folks (looks gcc-compilerish to me, or maybe fallout from slight changes in C function call semantics). --Scott David Daniels Scott.Daniels@Acm.Org
It doesn't for me (CVS HEAD, OS X Panther).
Note sure what you mean "CVS HEAD", you mean the latest python from cvs? 2.4? I'm still using the Apple python, which is straight 2.3.
Have you, you know, tried to debug the situation yourself? If you have gcc installed, you probably have gdb installed too...
It's been around 7 years since I've used C, I've forgotten virtually everything I may have known about gdb, I've never worked with the C-python API before... meanwhile there is intense time pressure to get the next release of our product (http://www.goombah.com) ready. So it's just not practical for me to take that on myself now. I'm hoping to get some help from other pythonistas where someone will say -- "yes, it's getting a bus error for so-and-so reason, and if you do it this other way, you'll be fine..." Thanks, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson@goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net On Mon, 27 Jun 2005 21:56:44 +0100, Michael Hudson wrote:
Gary Robinson <grobinson@goombah.com> writes:
That caused a bus error 100% of the time when I simply imported the module into Python and called getSumChiSquare(), i.e.:
import testfloat testfloat.getSumChiSquare()
It doesn't for me (CVS HEAD, OS X Panther).
Could it be that this is a python bug? Or am I doing something wrong?
Note: this is using Python 2.3 on OS X:
Python 2.3 (#1, Sep 13 2003, 00:49:11)
Thanks in advance for any help or insight you can give,
Have you, you know, tried to debug the situation yourself? If you have gcc installed, you probably have gdb installed too...
Cheers, mwh
Gary Robinson wrote:
#include <Python.h> static double gfSumChiSquare = 123.0;
static PyObject * getSumChiSquare(PyObject *self, PyObject *args){ return Py_BuildValue("d", gfSumChiSquare); }
static PyMethodDef SimMethods[] = { {"getSumChiSquare", getSumChiSquare, METH_NOARGS, "Return fSumChiSquare"}, {NULL, NULL, 0, NULL} /* Sentinel */ };
PyMODINIT_FUNC inittestfloat(void) { (void) Py_InitModule("testfloat", SimMethods); }
Could it be that this is a python bug? Or am I doing something wrong?
You are doing something wrong. As getSumChiSquare is a METH_NOARGS method, it shouldn't have a PyObject*args argument. However, python-dev really isn't the place to get urgent help if "there is intense time pressure to get the next release of our product (http://www.goombah.com) ready". Instead, if you need help in this case, you should hire somebody. Regards, Martin
You are doing something wrong. As getSumChiSquare is a METH_NOARGS method, it shouldn't have a PyObject*args argument.
While I am aware enough of your general expertise to be surprised if you were in error in your statement below about VAR_NOARGS, your statement obove does seem to contradict the docs: METH_VARARGS This is the typical calling convention, where the methods have the type PyCFunction. The function expects two PyObject* values. The first one is the self object for methods; for module functions, it has the value given to Py_InitModule4() (or NULL if Py_InitModule() was used). The second parameter (often called args) is a tuple object representing all arguments. This parameter is typically processed using PyArg_ParseTuple() or PyArg_UnpackTuple. ... METH_NOARGS Methods without parameters don't need to check whether arguments are given if they are listed with the METH_NOARGS flag. They need to be of type PyCFunction. When used with object methods, the first parameter is typically named self and will hold a reference to the object instance. In all cases the second parameter will be NULL. ---- (http://python.fyxm.net/doc/2.3.5/api/common-structs.html#l2h-832 In other words the docs seem to explicitly state that the args will be supplied and what they will contain, so I'd assume that the C method should have them in their declaration. Moreover, METH_NOARGS needs to correspond to type PyCFunction, and the definition of type PyCFunction states that it must receive two args of PyObject*. Are the docs wrong or am I misreading them? Or are you wrong? Re your other comments, as mentioned in my personal email, the word done by folks in a couple of python forums in running tests does seem to confirm that there is some kind of bug involving gcc 3.3. The bus error appears independently of whether VAR_METHARGS or VAR_VARARGS is used. Also, (again as mentioned in my personal email) I do have 4 F/T experienced people working on this project, none of whom has the expertise to resolve this. Given that it looks like a real bug was uncovered, I think it was appropriate to post on the python forums about this. I only posted to python-dev because it was suggested that I do so by someone on the C++ sig. Gary -- Gary Robinson CTO Emergent Music, LLC grobinson@goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net
Gary Robinson wrote:
Are the docs wrong or am I misreading them? Or are you wrong?
It turns out that I am wrong. The NOARGS functions are indeed called with an additional NULL argument; it's just that many functions with NOARGS in Python itself are declared without the additional argument. Regards, Martin
"Martin v. Löwis" <martin@v.loewis.de> writes:
Gary Robinson wrote:
Are the docs wrong or am I misreading them? Or are you wrong?
It turns out that I am wrong.
This is a long standing confusion. At one point, the documentation said what you said, and it was just as wrong. There were even inaccurate PyNoArgsCFunction typedefs in some header! I think Python in a Nutshell is wrong here too.
The NOARGS functions are indeed called with an additional NULL argument; it's just that many functions with NOARGS in Python itself are declared without the additional argument.
I've been fixing these when I find them, slowly. Fortunately (and obviously, given the lack of bug reports) it won't make any difference with any ABI I know about. Cheers, mwh -- I have gathered a posie of other men's flowers, and nothing but the thread that binds them is my own. -- Montaigne
participants (4)
-
"Martin v. Löwis" -
Gary Robinson -
Michael Hudson -
Scott David Daniels