AW: [pypy-dev] multiple isolated VMs in a single process

[Tobias Oberstein]

Hello Holger,

> > > > I'm thinking of embedding a dynamic language into an
> > > > multithreaded OODBMS. The different VMs within the server process
> > > > would share data via objects instantiated from a special
> > > > extension class that wraps up the OODBMS. Synchronisation on
> > > > these objects is taken care of by the OODBMS. No need to share
> > > > objects local to each VM.
> > >
> > > A valid use case, IMO.  although some people would say
> > > that you might use ZODB which does it with multiple
> > > processes.   To me the biggest point for using processes
> > > is stability.  It does come at a cost, though.  You have
> > > to deal with "copies" of objects and only have "shared"
> > > objects at a higher level.  This might suit your needs, though.
> >
> > It imposes a big performance hit, since every call on an object
> > from a persistable class must go over IPC, unless the object is
> > cached in the process.
>
> Which is the whole point of ZEO/RPC.  ZODB doesn't offer an
> RPC mechanism between python objects (IIRC).
>
> > But then again all issues of synchronization
> > (be it pessimistic via locking or optimistic via multi-versioning)
> > arise. I don't like it.
>
> I see your point but there are locking issues with multithreading,
> too.

Yes, but it's solved (hopefully) by a robust OODBMS. By the way, speaking
of "robust", probably I should just reveal a bit of background info why
I'm so nuts about this stuff (it's not strictly on topic and got long,
but eh ..):

In 2000, SAP AG open-sourced a enterprise grade RDBMS (it runs R/3 so it's
no toy stuff): http://www.sapdb.org

What few people know is the fact that SAP also (yet unofficially) released
the "LiveCache"-technology as open-source. LiveCache is a combination of
an OODBMS integrated on top of SAP DB (which makes SAP DB a _hybrid_
database)
plus a COM-like run-time environment that makes it possible to call
components
implemented in C++ from database client sessions just like any other stored
procedure.

Components are run within the address space (in-process) of LiveCache (which
is multi-threaded) for reason of maximum performance. SAP currently ship
LiveCache as the basis for SAP-APO, which is part of SAP's supply-chain mgt.
system. They developed it, because APO must perform in real-time. Ah, and
there are LiveCache installations on SMPs with 64 GB Ram and > 20 CPUs.

I hope this all explains my interest: "Python embedded in LiveCache"
I bet this could be a real killer app! Real enterprise grade integrated
application server/OODBMS with modern high-level dynamic language.
Btw: I'm NOT employed by SAP or in any way related with them.

>
> > Regarding the stability issue: agreed when speaking of components
> > hacked in C/C++. But if I have components in Python, then I don't
> > see any problems unless my VM is buggy of course.
>
> sure but if there are IO problems in some tasks then it's nice
> to be able to 'signal' them in a clean way.  Threads don't
> mix well with asynchronous signals.

The only IO I'd like to do is via persisted objects. It's pretty much
a "sandbox"-Python I'm thinking of. More radically: "import" should
look in the database for code, not in the filesystem. Maybe I'm better
off taking a minimal subset of CPython and tweaking it until it fits.

> But you didn't answer the Sprint question :-)
> i presume you can't make time on such a short notice.

Problem is, instead of writing postings I should learn for the tests
I have to do in the coming 2 months (I'm studying math) .. well, I'm
an addict geek. Anyway, PyPy is way cool and I guess the Sprint will
be great!

Greets, Tobias