Python Performance vs. C++ in a Complex System

[Boudewijn Rempt]

Courageous <jkraska1 at san.rr.com> wrote:
> On Mon, 16 Apr 2001 16:43:42 GMT, ben at co.and.co wrote:

> You're exactly correct. In C++, I'd be completely toasted if I were using
> kernel-level threads. However, I'm not. I'm doing cooperative multithreading
> on top of exactly one OS thread. Preemptive multitasking is exactly what I
> don't want: this is a simulation where time-slicing across real-time is not meaningful.
> It is the coordinated flow of agent-tasks through SIMULATED time which matters.

> My cooperative multithreading core is a highly-optimized vector-allocated
> priority heap; context-switches are essentially a register save/restore, however
> this is accomplished with Microsoft's Fiber interface, which allows me to avoid
> assembly. My prior limit here is the minimum 1 page (64K) commit of virtual
> memory per thread; with a user-addressable virtual memory under 2K/NT
> of 2 gigs, the C++ system drops dead at just over 31,000 threads.

> The Python implementation can handle considerably more, and it's largely
> dependent on real memory only (no virtual limit). This is because Stackless
> has only one real stack (in the interpreter).


I find the number of 30.000 extremely interesting. I'm currently
engaged in building a commercial autonomous agent platform in Java. The
quite unoptimized (we feel - although we spent a lot of time on it)
co-operative multitasker we built could handle about 30.000 complex
agents, too, on a four-processor Dell box. With more, we started to get
timing problems. This was with with a commercial trading application,
built in a few months. At JavaOne we hope to be doling out CD's with the
lite version of the toolkit ;-). I cannot help wondering whether 30.000
is a magic boundary of some kind.

This contrasts to an earlier approach, Gossip, which followed all
the textbook advice about giving one thread to each agent.  For both
the toolkit and gossip (which is free, but closed source), see
http://www.tryllian.com.

I'm currently experimenting with stackless python to see how well that
compares to Java.

-- 

Boudewijn Rempt  | http://www.valdyas.org | Christos Voskrese!