Python probe: execute code in isolation (subinterpreter?) and get results
Hi, Is there a standard way to execute a Python code and inspect the results without spawning an external Python process? If there is no such way, I'd like to propose the feature, and there are two user stories. Both are about application environment probing. Story #1: Choosing the best library in a safe manner Probing environment is required for Python applications to make component selection logic explicit and less error-prone. I can tell from my experience with Spyder IDE that startup procedure is the most fragile part for this cross-platform application, which makes use of optionally installed components on user system. Implicit import nature and inability to revert import operation makes situation complicated. Below is an example. Take a note that this is not about packaging. Spyder IDE is a Qt application that optionally embeds IPython console. Qt has two bindings - PyQt4 and PySide. PyQt4 binding has two APIs - #1 and #2. If PyQt4 is used and version of installed IPython >= 0.11, the API #2 must be chosen. So, the IPython version probing should come first. A standard way to detect IPython version is to import IPython before the rest of the application, but IPython may detect PyQt4 itself and import it too for probing version. And if Spyder uses PySide we now have a conflict with Qt libraries loaded. If there was a way to execute Python script in subinterpreter to probe all installed component versions and return results, the selection logic would be much more readable and sane. Story #2: Get settings from user script Blender uses SCons to automate builds. SCons script is written in Python and it uses execfile(filename, globals, locals) to read platform specific user script with settings. Unfortunately, execfile() is a hack that doesn't treat Python scripts the same way the interpreter treats them - for example, globals access will throw exceptions - http://bugs.python.org/issue14049 More important that users won't be able to troubleshoot the exceptions, because standalone script works as expected. Executing user script code in a subprocess will most likely negatively affect performance, which is rather critical for a build tool. Pickling and unpickling objects with global state through communication pipe may also be the source of bugs. So, having a cheap way to communicate with Python subinterpreter and get a simple dict in result will make Python more snappy. -- anatoly t.
anatoly techtonik wrote:
Given that you are also loading external shared libraries, I don't see how you could do this within the same process. Unloading shared libs is possible (even if fragile), but if you don't even know which libs have been loaded, likely impossible to do in a cross- platform way.
You're not using execfile() correctly: if you want a script to be run in the same way as a module, then the local and global namespace dictionaries have to be the same. So the second story already works in vanilla Python :-) Lots of Python applications read configuration data from user supplied (Python) config files. It's less secure than e.g. INI files, but gives you a lot of flexibility in defining what you need.
I don't see how you could get story #1 working without a subprocess. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 04 2012)
2012-04-03: Python Meeting Duesseldorf today ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 4/4/2012 4:25 AM, anatoly techtonik wrote:
Please stop misrepresenting Python. I clearly explained the issue in 14049 before closing it. If you did not understand, re-read until you do. Trying to get an object via an unbound name (non-existent variable), always results in a NameError. There is nothing unique about execfile. If Blender's scons script is using execfile wrong (by passing separate globals and locals), tell them to fix *their* bug. -- Terry Jan Reedy
anatoly techtonik wrote:
Given that you are also loading external shared libraries, I don't see how you could do this within the same process. Unloading shared libs is possible (even if fragile), but if you don't even know which libs have been loaded, likely impossible to do in a cross- platform way.
You're not using execfile() correctly: if you want a script to be run in the same way as a module, then the local and global namespace dictionaries have to be the same. So the second story already works in vanilla Python :-) Lots of Python applications read configuration data from user supplied (Python) config files. It's less secure than e.g. INI files, but gives you a lot of flexibility in defining what you need.
I don't see how you could get story #1 working without a subprocess. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 04 2012)
2012-04-03: Python Meeting Duesseldorf today ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 4/4/2012 4:25 AM, anatoly techtonik wrote:
Please stop misrepresenting Python. I clearly explained the issue in 14049 before closing it. If you did not understand, re-read until you do. Trying to get an object via an unbound name (non-existent variable), always results in a NameError. There is nothing unique about execfile. If Blender's scons script is using execfile wrong (by passing separate globals and locals), tell them to fix *their* bug. -- Terry Jan Reedy
participants (3)
-
anatoly techtonik -
M.-A. Lemburg -
Terry Reedy