capi-sig September 2008

capi-sig@python.org

9 participants
6 discussions

ANNOUNCE: PyBindGen 0.8
by Gustavo Carneiro 10 Mar '22

10 Mar '22

The other day I released PyBindGen 0.8. Main news is that it features a new experimental header file scanner based on pygccxml (i.e., it's similar to py++ in scope, if not in maturity, but does not use boost.pythonunderneath). == What == PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write, and does not depend on any library or header files besides Python itself. == Where == https://launchpad.net/pybindgen/ == NEWS == - Support C++ instance attributes through getter/setter methods - Support functions as methods of C++ classes - Support the PyObject* type - Support unsigned int, C strings (char*) (from Mark Lee) - Add basic support for enum types - New experimental automatic module generator based on C/C++ header file scanner and annotations in comments, using pygccxml - Some bug fixes -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert

2 1

Embedded Python in C application
by Eljay Love-Jensen 27 Sep '08

27 Sep '08

Hi everyone, First, my apologies if I'm in the wrong forum for my "embedding Python in a C application" questions. Please redirect me if I've wandered into the wrong place. I have two needs for using Python in my application that I hope has an easy answer without rewriting Python's internals. I need to use Python* in a multi-threaded application, where separate threads may be working on very long lasting Python scripts, and other threads may be involved in short Python scripts. None of the Python scripts running concurrently have any shared state with any of the other Python scripts running concurrently. Number of threads is in the 100-1000 range. I need to manage Python's use of the heap by providing a memory pool for Python to use, rather than allowing Python to use malloc/free. This is to prevent memory fragmentation, and to allow easy disposal of a memory pool used for a closed Python interpreter instance. A quick view of Py_Initialize() indicates that Python does not return some sort of "Py_State" pointer which represents the entire state of a Python interpreter. (Nor some sort of Py_Alloc().) Nor accepts a custom malloc/free function pointers. Hmmm. Does anyone have experience with using Python in this fashion? (If relevant, it will be Python 3.x not Python 2.x.) Thanks, --Eljay * Doesn't HAVE to be Python. Could be JavaScript or Lua or whatnot. My preferences these days is a Python solution.

5 6

Overriding __builtins__ from C to sandbox python scripts
by Campbell Barton 06 Sep '08

06 Sep '08

This issue has come up on blender3d since there is work on bringing back the web plugin and also came up on the pygame list. Blender3D currently uses the C api to override __builtin__ functions that could be used to to bad stuff. so you cant do... "import os; os.system('rm -rf ~')" The builtins overridden are import, open, file, execfile, compile and reload. Keep in mind this is not intended to run arbitrary python script, They are packed into a file, external scripts cant be accessed. so the scripts will just have to cope with running inside a restricted python env. By question is - Is there a way to get back the original functions that the C api over writes when python is initialized? - If not, is there any way to run "import os; os.system('rm -rf ~')" from this limited environment? If this is the case then ofcourse there are other problems, " " * 1000 * 1000 * 1000 * 1000 * 1000 * 1000 - would crash the plugin... but for now Im mainly interested in worst case where any py code can be run that damages the system. Here is blender3d's sandbox function with some junk removed to read better. ------------------ // Python Sandbox code // override builtin functions import() and open() PyObject *KXpy_open(PyObject *self, PyObject *args) { PyErr_SetString(PyExc_RuntimeError, "Sandbox: open() function disabled!\nGame Scripts should not use this function."); return NULL; } PyObject *KXpy_reload(PyObject *self, PyObject *args) { PyErr_SetString(PyExc_RuntimeError, "Sandbox: reload() function disabled!\nGame Scripts should not use this function."); return NULL; } PyObject *KXpy_file(PyObject *self, PyObject *args) { PyErr_SetString(PyExc_RuntimeError, "Sandbox: file() function disabled!\nGame Scripts should not use this function."); return NULL; they are packaged inside a blend file so no other scripts will run. } PyObject *KXpy_execfile(PyObject *self, PyObject *args) { PyErr_SetString(PyExc_RuntimeError, "Sandbox: execfile() function disabled!\nGame Scripts should not use this function."); return NULL; } PyObject *KXpy_compile(PyObject *self, PyObject *args) { PyErr_SetString(PyExc_RuntimeError, "Sandbox: compile() function disabled!\nGame Scripts should not use this function."); return NULL; } PyObject *KXpy_import(PyObject *self, PyObject *args) { char *name; PyObject *globals = NULL; PyObject *locals = NULL; PyObject *fromlist = NULL; PyObject *l, *m, *n; if (!PyArg_ParseTuple(args, "s|OOO:m_import", &name, &globals, &locals, &fromlist)) return NULL; /* check for builtin modules */ m = PyImport_AddModule("sys"); l = PyObject_GetAttrString(m, "builtin_module_names"); n = PyString_FromString(name); if (PySequence_Contains(l, n)) { return PyImport_ImportModuleEx(name, globals, locals, fromlist); } /* quick hack for GamePython modules TODO: register builtin modules properly by ExtendInittab */ if (!strcmp(name, "GameLogic") || !strcmp(name, "GameKeys") || !strcmp(name, "PhysicsConstraints") || !strcmp(name, "Rasterizer") || !strcmp(name, "Mathutils")) { return PyImport_ImportModuleEx(name, globals, locals, fromlist); } PyErr_Format(PyExc_ImportError, "Import of external Module %.20s not allowed.", name); return NULL; } static PyMethodDef meth_open[] = {{ "open", KXpy_open, METH_VARARGS, "(disabled)"}}; static PyMethodDef meth_reload[] = {{ "reload", KXpy_reload, METH_VARARGS, "(disabled)"}}; static PyMethodDef meth_file[] = {{ "file", KXpy_file, METH_VARARGS, "(disabled)"}}; static PyMethodDef meth_execfile[] = {{ "execfile", KXpy_execfile, METH_VARARGS, "(disabled)"}}; static PyMethodDef meth_compile[] = {{ "compile", KXpy_compile, METH_VARARGS, "(disabled)"}}; static PyMethodDef meth_import[] = {{ "import", KXpy_import, METH_VARARGS, "our own import"}}; void setSandbox(TPythonSecurityLevel level) { PyObject *m = PyImport_AddModule("__builtin__"); PyObject *d = PyModule_GetDict(m); // functions we cant trust PyDict_SetItemString(d, "open", PyCFunction_New(meth_open, NULL)); PyDict_SetItemString(d, "reload", PyCFunction_New(meth_reload, NULL)); PyDict_SetItemString(d, "file", PyCFunction_New(meth_file, NULL)); PyDict_SetItemString(d, "execfile", PyCFunction_New(meth_execfile, NULL)); PyDict_SetItemString(d, "compile", PyCFunction_New(meth_compile, NULL)); // our own import PyDict_SetItemString(d, "__import__", PyCFunction_New(meth_import, NULL)); }

1 0

PyImport_ImportModule and folders
by anti00Zero 03 Sep '08

03 Sep '08

Hello. I have a problem with importing a script in a folder. The script is in a folder simple\index.py. This folder is in the root folder of my application. I use boost 1.33.1 with Python 2.4. Here is my code: PyObject *pObject = PyImport_ImportModule(const_cast<char*>(pcSkriptName.c_str())); If I use pcSkriptName with value "index" all works right. Now I want to load this script in the folder simple. But pcSkriptName with value "simple.index" doesn't work. What shell I do? Thanks for responding. -- Psssst! Schon das coole Video vom GMX MultiMessenger gesehen? Der Eine für Alle: http://www.gmx.net/de/go/messenger03

2 1

Threading in Python
by Swapnil.ST＠gmail.com 02 Sep '08

02 Sep '08

I am working on an application which embeds python in the application threads and each thread can further create threads using the python thread module few things i don't understand: 1. why does the interpreter ie the main thread keeps releasing and reacquiring the GIL every few instruction. why isn't GIL simply allowed to be acquired by any thread which requires it then. I mean why is the acquisition and release of a lock kept periodic and not on need basis?( which is obviously the way it should be) 2. The purpose of GIL was not very clear to me from python-docs. is it just a single lock to guard every global variable? or is it something more? 3. suppose I am not making any blocking call then I would never release GIL since that is the only thing guarding my thread state and i don't want my thread state messed up any time my thread is running. Which implies unacceptable starvation of other threads.

3 2

Threading in Python
by Swapnil.ST＠gmail.com 02 Sep '08

02 Sep '08

Hi Gustavo Thanks for reply I am trying to test the thread module that I have put on top of application thread with a simple script that creates two threads. I am getting the following error, the cause of error I cannot understand. The same script when run with bare python, runs perfectly well Traceback (most recent call last): Unhandled exception in thread started by Assertion failed: PyTuple_Check(mro), file .\Python\Objects\typeobject.c, line 824 File "thread_test.py", line 28, in <module> mythread = thread.start_new_thread(func1,tuple) SystemError: error return without exception set ERROR: Failed to import Python module thread_test!

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

capi-sig September 2008