Calling Methods from Pythons C API with Keywords
Hey Guys, My first post on this list so I hope this is the right place to post and relevant. Im rewriting parts of the Blender3D python API that has got a bit old and needs an update. Im making a PyList subtype with the C/Python API, this involves intercepting calls to standard list methods to make sure Blenders array data is in Sync with the list's data. Iv got it working for tp_as_sequence, tp_as_mapping, iter and dealloc etc but methods are a problem. I want to add my own call's before and after PyLists standard functions but have a proplem with functons that use keywords and have no API equivalent. For example, I cant use the API's PyList_Sort because that dosnt support keywords like... ls.sort(key=lambda a: a.foo)) And the Problem with PyObject_CallMethod is that it dosnt accept keywords. PyObject_CallMethod((PyObject *)mylist, "sort", "O", args); Looking at abstract.c, PyObject_CallMethod uses call_function_tail, which calls "PyObject_Call(callable, args, NULL);" - so Its not currently possible with PyObject_CallMethod. But I cant find any way to do this in a few lines. I could use PyEval_CallObjectWithKeywords but that would mean Id need to get the method from the list manually which Ill look into, but unless Im missing something here, it seems PyObject_CallMethodWithKeywords would be a nice addition to the Python API that cant be done in a straight forward way at the moment. - Thanks
On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
I want to add my own call's before and after PyLists standard functions but have a proplem with functons that use keywords and have no API equivalent. For example, I cant use the API's PyList_Sort because that dosnt support keywords like...
ls.sort(key=lambda a: a.foo))
And the Problem with PyObject_CallMethod is that it dosnt accept keywords.
Note that you can always simply call PyObject_Call on the bound method object retrieved using PyObject_GetAttrString. The hardest part is usually constructing the keywords dictionary, a job best left to Py_BuildValue and friends. When I need that kind of thing in more than one place, I end up with a utility function like this one: /* Equivalent to PyObject_CallMethod but accepts keyword args. The format... arguments should produce a dictionary that will be passed as keyword arguments to obj.method. Usage example: PyObject *res = call_method(lst, "sort", "{s:O}", "key", keyfun)); */ PyObject * call_method(PyObject *obj, const char *methname, char *format, ...) { va_list va; PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL; args = PyTuple_New(0); if (!args) goto out; meth = PyObject_GetAttrString(obj, methname); if (!meth) goto out; va_start(va, format); kwds = Py_VaBuildValue(format, va); va_end(va); if (!kwds) goto out; ret = PyObject_Call(meth, args, kwds); out: Py_XDECREF(meth); Py_XDECREF(args); Py_XDECREF(kwds); return ret; } It would be nice for the Python C API to support a more convenient way of calling objects and methods with keyword arguments.
Hrvoje Nikšić wrote:
On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
I want to add my own call's before and after PyLists standard functions but have a proplem with functons that use keywords and have no API equivalent. For example, I cant use the API's PyList_Sort because that dosnt support keywords like...
ls.sort(key=lambda a: a.foo))
And the Problem with PyObject_CallMethod is that it dosnt accept keywords.
Note that you can always simply call PyObject_Call on the bound method object retrieved using PyObject_GetAttrString. The hardest part is usually constructing the keywords dictionary, a job best left to Py_BuildValue and friends. When I need that kind of thing in more than one place, I end up with a utility function like this one:
/* Equivalent to PyObject_CallMethod but accepts keyword args. The format... arguments should produce a dictionary that will be passed as keyword arguments to obj.method.
Usage example: PyObject *res = call_method(lst, "sort", "{s:O}", "key", keyfun)); */
PyObject * call_method(PyObject *obj, const char *methname, char *format, ...) { va_list va; PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL;
args = PyTuple_New(0); if (!args) goto out; meth = PyObject_GetAttrString(obj, methname); if (!meth) goto out;
va_start(va, format); kwds = Py_VaBuildValue(format, va); va_end(va); if (!kwds) goto out;
ret = PyObject_Call(meth, args, kwds); out: Py_XDECREF(meth); Py_XDECREF(args); Py_XDECREF(kwds); return ret; }
It would be nice for the Python C API to support a more convenient way of calling objects and methods with keyword arguments.
Thanks for the hint, I ended up using PyObject_Call. This seems to work, EXPP_PyTuple_New_Prepend - is a utility function that returns a new tuple with self at the start (needed so args starts with self) I dont think I can use PyObject_GetAttrString because the subtype would return a reference to this function - rather then the lists original function, Id need an instance of a list and dont have one at that point. ______________________ static PyObject * MaterialList_sort(BPy_MaterialList *self, PyObject *args, PyObject *keywds ) { PyObject *ret; PyObject *newargs = EXPP_PyTuple_New_Prepend(args, (PyObject *)self); sync_list_from_materials__internal(self); # makes sure the list matches blenders materials ret = PyObject_Call(PyDict_GetItemString(PyList_Type.tp_dict, "sort"), newargs, keywds); Py_DECREF(newargs); if (ret) sync_materials_from_list__internal(self); # makes blenders materials match the lists return ret; } _____________________ Later on Ill probably avoid using PyDict_GetItemString on PyList_Type.tp_dict all the time since the methods for lists does not change during python running. - Can probably be assigned to a constant. -- Campbell J Barton (ideasman42)
[ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ] On Wed, 2007-06-20 at 20:17 +1000, Campbell Barton wrote:
I dont think I can use PyObject_GetAttrString because the subtype would return a reference to this function - rather then the lists original function, Id need an instance of a list and dont have one at that point.
Note that PyList_Type is a full-fledged PyObject, so PyObject_GetAttrString works on it just fine. Of course, you would also need to add the "self" argument before the keywords, but that's a trivial change to the function. Calling PyObject_GetAttrString feels cleaner than accessing tp_dict directly, and most importantly call_method as written delegates creation of the dictionaty to Py_BuildValue.
Hrvoje NikÅ¡iÄ wrote:
[ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ]
Is there a list for this kind of discussion? Iv tried asking questions on the freenode python chat room but almost very few people there do C/Python api development. -- Campbell J Barton (ideasman42)
On Wed, 2007-06-20 at 22:12 +1000, Campbell Barton wrote:
Hrvoje Nikšić wrote:
[ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ]
Is there a list for this kind of discussion?
I believe the appropriate list would be the general Python list/newsgroup. I agree that response about the Python/C API tends to be sparse on general-purpose lists, though. If there is a forum dedicated to discussing the *use* of Python at the C level, I'd like to know about it as well.
Campbell Barton schrieb:
Hrvoje NikÅ¡iÄ wrote:
[ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ]
Is there a list for this kind of discussion?
Hrvoje wasn't explicit on *why* this discussion is inappropriate here, so I just add that for better understanding: python-dev is for the development *of* Python, not for the development *with* Python. So you post here if you propose an enhancement or discuss the resolution of a bug. Question of the "how do I" kind are off-topic - posters are expected to know and understand the options, and then discuss the flaws of these options, rather than asking what they are. As Hrvoje says: try python-list (aka comp.lang.python). If you don't get an answer, you didn't phrase your question interestingly enough, or nobody knows the answer, or nobody has the time to tell you. Regards, Martin
On Wed, 2007-06-20 at 19:26 +0200, "Martin v. Löwis" wrote:
As Hrvoje says: try python-list (aka comp.lang.python). If you don't get an answer, you didn't phrase your question interestingly enough, or nobody knows the answer, or nobody has the time to tell you.
The thing with comp.lang.python is that it is followed by a large number of Python users, but a much smaller number of the C API users -- which is only natural, since the group is about Python, not about C. For most users the Python/C API is an implementation detail which they never have to worry about. Futrhermore, questions about the C API often concern CPython implementation details and so they don't feel like they would belong in comp.lang.python. As an experiment, it might make sense to open a mailing list dedicated to the Python C API. It could become a useful support forum for extension writers (a group very useful to Python) and maybe even a melting pot for new ideas regarding CPython, much like comp.lang.python historically provided ideas for Python the language.
The reason I asked on this in the first place is I had looked through the python source to make sure PyObject_Call had no equivalent that supported keywords, and since I needed to do this I figured it might be worth considering for Pythons API. Im sure everyone could write their own PyObject_Call, if they had to but thats what the API's for. Hrvoje Nikšić wrote:
On Wed, 2007-06-20 at 19:26 +0200, "Martin v. Löwis" wrote:
As Hrvoje says: try python-list (aka comp.lang.python). If you don't get an answer, you didn't phrase your question interestingly enough, or nobody knows the answer, or nobody has the time to tell you.
The thing with comp.lang.python is that it is followed by a large number of Python users, but a much smaller number of the C API users -- which is only natural, since the group is about Python, not about C. For most users the Python/C API is an implementation detail which they never have to worry about.
Futrhermore, questions about the C API often concern CPython implementation details and so they don't feel like they would belong in comp.lang.python. As an experiment, it might make sense to open a mailing list dedicated to the Python C API. It could become a useful support forum for extension writers (a group very useful to Python) and maybe even a melting pot for new ideas regarding CPython, much like comp.lang.python historically provided ideas for Python the language.
Agree a Python/C API List would be great, in fact I cant see any reasons not to have it- likely the pure python users dont want to know about refcounting problems.. etc anyway. http://mail.python.org/mailman/listinfo http://www.python.org/community/sigs/ There are lists/newsgroup for py2exe and pyrex, Python-ObjectiveC etc, Python/C API seems much more generic, and its also fairly tricky to use at times - when doing more advanced stuff (subtyping has been tricky for me anyway). I expect the dev's of pyrex, pygame etc might also need to discuss C API spesific issues as well. Iv had roughly this conversation in IRC... Q. Hi, Id like to know how wrap python subtype methods in the C API A. C dosnt have classes, use C++ Q. no I want to use pythons C API, A. Subtypes are easy to do in python.. ...... you get the idea... Quite a few "python only" users dont understand where the Python/C API fits in and its annoying to have to explain the question each time (yes, Iv had these conversations more then once) - Cam
Futrhermore, questions about the C API often concern CPython implementation details and so they don't feel like they would belong in comp.lang.python. As an experiment, it might make sense to open a mailing list dedicated to the Python C API. It could become a useful support forum for extension writers (a group very useful to Python) and maybe even a melting pot for new ideas regarding CPython, much like comp.lang.python historically provided ideas for Python the language.
In the past, we created special-interest groups for such discussion. Would you like to coordinate a C sig? See http://www.python.org/community/sigs/ Regards, Martin
participants (3)
-
"Martin v. Löwis"
-
Campbell Barton
-
Hrvoje Nikšić