Hi everyone,
First, my apologies if I'm in the wrong forum for my "embedding Python in a
C application" questions. Please redirect me if I've wandered into the
wrong place.
I have two needs for using Python in my application that I hope has an easy
answer without rewriting Python's internals.
I need to use Python* in a multi-threaded application, where separate
threads may be working on very long lasting Python scripts, and other
threads may be involved in short Python scripts. None of the Python scripts
running concurrently have any shared state with any of the other Python
scripts running concurrently. Number of threads is in the 100-1000 range.
I need to manage Python's use of the heap by providing a memory pool for
Python to use, rather than allowing Python to use malloc/free. This is to
prevent memory fragmentation, and to allow easy disposal of a memory pool
used for a closed Python interpreter instance.
A quick view of Py_Initialize() indicates that Python does not return some
sort of "Py_State" pointer which represents the entire state of a Python
interpreter. (Nor some sort of Py_Alloc().) Nor accepts a custom
malloc/free function pointers. Hmmm.
Does anyone have experience with using Python in this fashion?
(If relevant, it will be Python 3.x not Python 2.x.)
Thanks,
--Eljay
* Doesn't HAVE to be Python. Could be JavaScript or Lua or whatnot. My
preferences these days is a Python solution.
This issue has come up on blender3d since there is work on bringing
back the web plugin and also came up on the pygame list.
Blender3D currently uses the C api to override __builtin__ functions
that could be used to to bad stuff.
so you cant do...
"import os; os.system('rm -rf ~')"
The builtins overridden are import, open, file, execfile, compile and reload.
Keep in mind this is not intended to run arbitrary python script, They
are packed into a file, external scripts cant be accessed. so the
scripts will just have to cope with running inside a restricted python
env.
By question is -
Is there a way to get back the original functions that the C api over
writes when python is initialized?
- If not, is there any way to run "import os; os.system('rm -rf ~')"
from this limited environment?
If this is the case then ofcourse there are other problems, " " * 1000
* 1000 * 1000 * 1000 * 1000 * 1000 - would crash the plugin... but for
now Im mainly interested in worst case where any py code can be run
that damages the system.
Here is blender3d's sandbox function with some junk removed to read better.
------------------
// Python Sandbox code
// override builtin functions import() and open()
PyObject *KXpy_open(PyObject *self, PyObject *args) {
PyErr_SetString(PyExc_RuntimeError, "Sandbox: open() function
disabled!\nGame Scripts should not use this function.");
return NULL;
}
PyObject *KXpy_reload(PyObject *self, PyObject *args) {
PyErr_SetString(PyExc_RuntimeError, "Sandbox: reload() function
disabled!\nGame Scripts should not use this function.");
return NULL;
}
PyObject *KXpy_file(PyObject *self, PyObject *args) {
PyErr_SetString(PyExc_RuntimeError, "Sandbox: file() function
disabled!\nGame Scripts should not use this function.");
return NULL; they are packaged inside a blend file so no other
scripts will run.
}
PyObject *KXpy_execfile(PyObject *self, PyObject *args) {
PyErr_SetString(PyExc_RuntimeError, "Sandbox: execfile() function
disabled!\nGame Scripts should not use this function.");
return NULL;
}
PyObject *KXpy_compile(PyObject *self, PyObject *args) {
PyErr_SetString(PyExc_RuntimeError, "Sandbox: compile() function
disabled!\nGame Scripts should not use this function.");
return NULL;
}
PyObject *KXpy_import(PyObject *self, PyObject *args)
{
char *name;
PyObject *globals = NULL;
PyObject *locals = NULL;
PyObject *fromlist = NULL;
PyObject *l, *m, *n;
if (!PyArg_ParseTuple(args, "s|OOO:m_import",
&name, &globals, &locals, &fromlist))
return NULL;
/* check for builtin modules */
m = PyImport_AddModule("sys");
l = PyObject_GetAttrString(m, "builtin_module_names");
n = PyString_FromString(name);
if (PySequence_Contains(l, n)) {
return PyImport_ImportModuleEx(name, globals, locals, fromlist);
}
/* quick hack for GamePython modules
TODO: register builtin modules properly by ExtendInittab */
if (!strcmp(name, "GameLogic") || !strcmp(name, "GameKeys") ||
!strcmp(name, "PhysicsConstraints") ||
!strcmp(name, "Rasterizer") || !strcmp(name, "Mathutils")) {
return PyImport_ImportModuleEx(name, globals, locals, fromlist);
}
PyErr_Format(PyExc_ImportError,
"Import of external Module %.20s not allowed.", name);
return NULL;
}
static PyMethodDef meth_open[] = {{ "open", KXpy_open, METH_VARARGS,
"(disabled)"}};
static PyMethodDef meth_reload[] = {{ "reload", KXpy_reload,
METH_VARARGS, "(disabled)"}};
static PyMethodDef meth_file[] = {{ "file", KXpy_file, METH_VARARGS,
"(disabled)"}};
static PyMethodDef meth_execfile[] = {{ "execfile", KXpy_execfile,
METH_VARARGS, "(disabled)"}};
static PyMethodDef meth_compile[] = {{ "compile", KXpy_compile,
METH_VARARGS, "(disabled)"}};
static PyMethodDef meth_import[] = {{ "import", KXpy_import,
METH_VARARGS, "our own import"}};
void setSandbox(TPythonSecurityLevel level)
{
PyObject *m = PyImport_AddModule("__builtin__");
PyObject *d = PyModule_GetDict(m);
// functions we cant trust
PyDict_SetItemString(d, "open", PyCFunction_New(meth_open, NULL));
PyDict_SetItemString(d, "reload", PyCFunction_New(meth_reload, NULL));
PyDict_SetItemString(d, "file", PyCFunction_New(meth_file, NULL));
PyDict_SetItemString(d, "execfile",
PyCFunction_New(meth_execfile, NULL));
PyDict_SetItemString(d, "compile", PyCFunction_New(meth_compile, NULL));
// our own import
PyDict_SetItemString(d, "__import__",
PyCFunction_New(meth_import, NULL));
}
Hello.
I have a problem with importing a script in a folder.
The script is in a folder simple\index.py. This folder is in the root folder of my application.
I use boost 1.33.1 with Python 2.4.
Here is my code:
PyObject *pObject = PyImport_ImportModule(const_cast<char*>(pcSkriptName.c_str()));
If I use pcSkriptName with value "index" all works right. Now I want to load this script in the folder simple.
But pcSkriptName with value "simple.index" doesn't work.
What shell I do?
Thanks for responding.
--
Psssst! Schon das coole Video vom GMX MultiMessenger gesehen?
Der Eine für Alle: http://www.gmx.net/de/go/messenger03
I am working on an application which embeds python in the application
threads and each thread can further create threads using the python thread
module
few things i don't understand:
1. why does the interpreter ie the main thread keeps releasing and
reacquiring the GIL every few instruction. why isn't GIL simply allowed to
be acquired by any thread which requires it then. I mean why is the
acquisition and release of a lock kept periodic and not on need basis?(
which is obviously the way it should be)
2. The purpose of GIL was not very clear to me from python-docs. is it just
a single lock to guard every global variable? or is it something more?
3. suppose I am not making any blocking call then I would never release GIL
since that is the only thing guarding my thread state and i don't want my
thread state messed up any time my thread is running. Which implies
unacceptable starvation of other threads.
Hi Gustavo Thanks for reply
I am trying to test the thread module that I have put on top of application
thread with a simple script that creates two threads. I am getting the
following error, the cause of error I cannot understand. The same script
when run with bare python, runs perfectly well
Traceback (most recent call last):
Unhandled exception in thread started by Assertion failed:
PyTuple_Check(mro), file .\Python\Objects\typeobject.c, line 824
File "thread_test.py", line 28, in <module>
mythread = thread.start_new_thread(func1,tuple)
SystemError: error return without exception set
ERROR: Failed to import Python module thread_test!
