[IronPython] Multiple engine instances in IP 2.0 and beyond (was IronPython 2.0 Alpha 1 Released)

[Dino Viehland]

Both you and Tony bring up a very good point.  I believe that we can solve the thread problem by using .NET's ExecutionContext.  I've never actually tried that, I just know it exists, so I'll need to look into that. That will cause the context to be flown to thread pool threads, newly created threads, and other various async points within the CLR.  One reason for its existence today is for security so the CLR is pretty good at making sure this always gets flowed (there are various ways to do operations which don't flow it, but they all have "Unsafe" in their name).

Based upon Tony's description of I'm starting to believe the swappable SystemStates rather than multiple PythonEngines are the right solution - especially given that CPython is using a similar technique and even more so if ExecutionContext solves the async/threading problem.

Thanks for all your feedback and let me know if you have any more.  I'll take it all to our discussion about the hosting APIs next week.

-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
Sent: Wednesday, May 02, 2007 12:19 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Multiple engine instances in IP 2.0 and beyond (was IronPython 2.0 Alpha 1 Released)

Dino Viehland wrote:
> I didn't even realize posting at the top was considered a no-no on mailing list etiquette...  But indeed, I am using Outlook.  I wonder if there's an option to change that somewhere...
>

I was a bit tongue in cheek - but top-posting suffers from the same
readability issues on mailing lists as it does in usenet posts.

> Thanks for your feedback Michael.  This thread has diverged a little bit so let me know if you have any thoughts on the rest of it.
>
It occurs to me that your suggestion won't work for us if the user
spawns any threads of their own (which is highly likely) - unless we can
tell what thread it was spawned from...

Meanwhile I'll try and digest the rest of *this* thread...

Michael


> -----Original Message-----
> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
> Sent: Wednesday, May 02, 2007 9:06 AM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Multiple engine instances in IP 2.0 and beyond (was IronPython 2.0 Alpha 1 Released)
>
> My guess is that you have to use outlook. It does seem to encourage
> top-posting. ;-)
>
> Dino Viehland wrote:
>
>> The scripts are running on multiple threads?
>>
> Usually - although sometimes code is executed on the GUI thread, but in
> this case we always know where to send the output.
>
>
>> The easy way to do this in v2.0 is to set console output (we no longer maintain our own output streams) to be a stream which looks at a thread static variable which is the real stream to output to.  Would that solve the entire isolation problem for you?
>>
>
>
> Our current code is nice and elegant, whilst yours sounds hacky. :-p
>
> Actually its just a solution we didn't think of, although it isn't quite
> as nice as running them in separate engines which we do now, and does
> give us *some* measure of isolation. (It is slightly less likely that a
> badly written user script will now kill the whole application, although
> obviously still very easy.)
>
>
>
>> The only problem w/ this is if user code sets sys.stdout they'll hijack all the other scripts.
>>
>>
>
> Which is a problem - but we can always say "don't do that then"...
>
> I think we still like our current use of multiple engines, and would
> prefer to see that supported in IronPython 2.0, even with a shared
> SystemState.
>
> At least you have given us an alternative without *having* to use
> AppDomains if this isn't possible.
>
> Thanks
>
> Michael Foord
> http://www.voidspace.org.uk/ironpython/index.shtml
>
>
>> -----Original Message-----
>> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
>> Sent: Wednesday, May 02, 2007 5:58 AM
>> To: Discussion of IronPython
>> Subject: Re: [IronPython] Multiple engine instances in IP 2.0 and beyond (was IronPython 2.0 Alpha 1 Released)
>>
>> Michael Foord wrote:
>>
>>
>>> Dino Viehland wrote:
>>>
>>>
>>>
>>>> I'm not actually the one working on the engine APIs so that's the reason I've tended to be vague.  I'll talk to the people doing it and let you know what I hear.
>>>>
>>>> But the more info you can give us the better decision we'll be able to make.  For example what do you actually need to be isolated?  Do you need multiple system states so they get their own modules, console, etc...  do you need everything in sys isolated?  Do you need to guarantee the isolation even if .NET code is called (e.g. they could smuggled data via a static field).  If they do need some rather high level of isolation are app domains good enough?  Do you need to marshal a lot of data in/out?  Or is the effort to spin up and use app domains correctly?
>>>>
>>>>
>>>>
>>>>
>>> At Resolver we are currently using multiple IronPython engines. Moving
>>> to AppDomains is a long term goal for us, but is actually quite a lot of
>>> work (we would have *lots* of cross-domain calls and so to avoid that we
>>> have to find an efficient way of pumping lots of data in and then out of
>>> the app domain).
>>>
>>> Switching to app domains is not a high priority task for us, and in the
>>> meantime we *can't* upgrade to IronPython 2 if it doesn't support
>>> multiple engines.
>>>
>>> Isolation of engines is only a minor benefit (it is a positive side
>>> effect - but not the reason we started using them) for us at the moment,
>>> and an isolated system state (although nice) is not vital.
>>>
>>>
>>>
>>>
>> In fact, the reason we started using multiple engines was to divert the
>> standard output of simultaneously running user scripts to different
>> output windows...
>>
>> Michael
>>
>>
>>
>>
>>> All the best,
>>>
>>> Michael Foord
>>> http://www.voidspace.org.uk/ironpython/index.shtml
>>>
>>> _______________________________________________
>>> users mailing list
>>> users at lists.ironpython.com
>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>
>>>
>>>
>>>
>> _______________________________________________
>> users mailing list
>> users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>> _______________________________________________
>> users mailing list
>> users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>
>>
>>
>
> _______________________________________________
> users mailing list
> users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> users mailing list
> users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>

_______________________________________________
users mailing list
users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com