[Python.NET] Embedding: finer control over module creation

[Tom Unger]

I did see the PythonScope object.  Your link helps explain it’s intended use:

  "eval" and "exec" are the most frequently used methods when interacting with Python.

However, that is not my use.  I have an application that relies heavily on scripting.  Initially I used IronPython and that has worked great for years.  However, many very useful modules don’t run on IronPython and development appears stalled.  So, time to move on.

My use has been to define an API for a specific operation and then implement that in a Python “script”.  These scripts function like modules but I call them “scripts” because they are not on the PYTHONPATH but are named specifically in my application configuration.  IronPython API lets me load, compile, and call the script explicitly.

This approach has been very successful and we have implemented quite complex applications in python scripts called by the .NET application.

Ideally, the my interface to CPython will use the same calling conventions.  Ideally, we will be able to change from IronPython to CPython (2.7) with no changes in the Python scripts.  So I’m trying to discover the python.net API equivalents to what I do in IronPython.  Many I have not found, and my impression is that my use of Python is different than other coders use it.  Above quote indicates this is true.  I have no intention to use eval and exec.  I want to load and compile code once at startup and then call repeatedly.

I have most of this working.  The one change I will have to check into python.net is addition of a call that loads a module from a file and associates the file name with that module.  This gets the file name into stack traces which will be very, very helpful when we come to debug our complex scripts.  It is a small feature addition.  I’ve written a test case for it.  I’ve not tested with Python 3 and should do that before creating the pull request.

Currently, I can not pre-load the module scope with pre-defined globals.  I load the module, it executes, and then I define the globals.  But most modules just define functions to be called by my application and don’t reference any of these variables in their “global scope code”.

I have found there are two significant difference from IronPython.  First is that CPython has one global scope.  In IronPython I was able to create a scope for each script that I loaded.  Any modules that script imported were within the scope of that script and entirely isolated from other scripts my application loaded.  CPython has one global scope.  If I load script A and script B and they both load module X, module X is shared.  Any changes on moduel X’s state will be seen by both A and B.   The isolation provided by IronPtyon was nice, but also mean that I could not write two scripts that coordinated logic with each other.  I never needed to do this.  The sharing in CPython will allow that.  However, it means that we must write any shared modules to assume multiple clients.  I think I have generally done this in all cases.  Testing will tell…

The other difference is that in IronPython, different scripts can execute concurrently on .net threads.  The scripts have their own scope, the data each works on is not touched by other threads, and all other parts of my application are thread safe.  My understanding is that CPython will not allow this.  It is a bit of loss in scalability but performance has never been our limiting factor.

That is a long description but I hope that some are interested in knowing how python.net is being used.

Tom


From: PythonDotNet <pythondotnet-bounces+tunger=mitem.com at python.org> On Behalf Of Denis Akhiyarov
Sent: Sunday, April 29, 2018 7:55 PM
To: A list for users and developers of Python for .NET <pythondotnet at python.org>
Cc: ywg <hbtmdxywg at 126.com>
Subject: Re: [Python.NET] Embedding: finer control over module creation

Tom,

You may be interested in this PyScope implementation merged into pythonnet in 2017. This is in master branch but not in any release yet.

https://github.com/pythonnet/pythonnet/pull/381

This was implemented by @yagweb (https://github.com/yagweb), who is now in core pythonnet team.

Regarding merging PR - if this small enough change with tests (if applicable) that don't require extensive review, then we can definitely merge it as long as it passes Travis CI and Appveyor CI.
If this is more intrusive PR, then at least few members of core team would need to review this.

BTW, if anyone is willing to step up with helping out on PR reviews, mailing list management, preparing releases, then please contact me.

Thanks,
Denis


On Fri, Apr 27, 2018 at 12:06 PM, Tom Unger (eVigils) <tunger at evigils.com<mailto:tunger at evigils.com>> wrote:
I’m starting on a project to embed CPython into a .NET application for scripting.  The application currently uses IronPython, which is falling behind.  Our scripting needs are fairly extensive and I’m challenged by the limited information I find available.  First questions are about some finer grain control over

First, my application loads a number of scripts.  I currently use PythonEngine.ModuleFromString() and this works but I would like finer control.


  1.  Set the filename in call to Py_CompileString().  Current this is set to “none”
  2.  Set globals for module before executing it.  I put certain values in the scripts global space for convenience.

PythonEngine.Compile() will compile the code but then I don’t see any way to execute the code.  Runtime.PyImport_ExecCodeModule(name, c); is not public.

Is there a way to do this with the current API?

If I fork the GIT repo, make changes that I need, are they likely to be incorporated?

Thanks,

Tom


_________________________________________________
Python.NET mailing list - PythonDotNet at python.org<mailto:PythonDotNet at python.org>
https://mail.python.org/mailman/listinfo/pythondotnet

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20180430/9b53934d/attachment-0001.html>