Python: performance, footprint, multi-threading, etc.

[Alexander Staubo]

In article <37458E5F.78B3A88D at tdv.com>, timb at tdv.com says...
> Hi,
> 
> I've just started looking into Python as a possible scripting language
> for a game we're developing.  The design of Python looks great, and the
> simplicity, portability, and embeddability (is that a word?) of it are
> what attracts me most.
[snip]

Python is great, but if your game depends on scripting performance, Python 
might not be such a great choice. It would depend on the measure of 
integration; would your scripts just do a "if GotKey: door.Open()" upon certain 
specific events? Or would your scripts consistute the game logic, drive the 
enemy AI and so forth? There's a marked difference between scripting an open-
ended game engine, and having scripts handle occasionally events driven by a 
game engine.

Python is generally agreed to be pretty slow. Some regard it as being in the 
order of 10-100 times slower than Java. Python is a high-level language 
designed to do high-level things, so the performance you'll squeeze out of the 
Python interpreter depends intrinsically upon the nature of your script. Lots 
of iteration and calculation, for example, is usually something that drags down 
the interpreter.

For raw speed, you might want to consider LCC, which is a C compiler that 
produces customizable code; essentially the code generator is wide open, so you 
can have it output anything, including proprietary bytecode; plug in your own 
portable bytecode interpreter and you have a fairly powerful scripting 
solution. The deep chasm that divides Python and C isn't just a philosophical 
one; it's painfully obvious that a dynamically-typed system such as Python will 
perform poorly, although for what it looses in speed it definitely catches up 
in flexibility and completeness.

Completeness is important. Python has a rather library of support modules. 
Serialization of game state is pretty simple given an efficient framework (eg., 
if all game state branches out of a single root object, storing the state to a 
file or a string takes just a single line of Python).

As for your multithreading statement: 50-100 threads? Read up on threading! If 
you're even considering loading this much on any scheduler, including NT, then 
you're not up to scratch. For example, if most of your threads are sleeping, 
why are they running in the first place? And even if 90% of your threads are 
sleeping, the context-switch hit and CPU load will bring your system to its 
knees very quickly. Add Python to this mess and you have a problem. NT has an 
extremely efficient scheduler, but it still doesn't solve the basic bottleneck 
that is the processor pipeline: It fits only one instruction at a time. With 
all this in mind, you should reconsider your design; usually you can get away 
with a single thread by using an event dispatching mechanism and a bit of 
clever thinking. Based on such a scheme, you can increase the number of threads 
dynamically (this is especially relevant on SMP systems) if need be. 
Furthermore not all platforms provide kernel-implemented threads; iirc Linux 
pre-2.0 only supports user-mode threads (cf. cooperative scheduling in Win3.x, 
and fibers on NT). The threading issue is not specific to Python, but to 
multitasking OSs in general. I don't know how Macs handle threading. Do any of 
the console systems support it? That's a pretty essential area of background 
research before even considering threading issues for a portable game engine. 
Anyway, to answer your question -- is this level of multi-threading not 
encouraged? -- you're right, it's not. You can pull it off, but why would you 
want to?

As for the size of the Python redistributable DLL, it will easily be dwarfed by 
the memory footprint of your actual Python scripts. If all you're running is 
"if key open door" scripts, then there's little need to worry. But if you're 
dealing with lots of data -- AI trees, model objects, maps, game state -- then 
you should start looking at Python's memory usage. It's not _bad_, in fact it's 
pretty good -- but worthy of inspection at any rate, lest you wake up one day 
with a scripting language that blows your memory requirements through the roof.

If performance is paramount, the only way for you to decide on a solution is to 
do test cases. Put together a prototype in Python -- for example, you could 
simulate a bunch of enemies fighting each other or something -- and see how it 
goes. Fortunately, Python permits integration with code written in other 
languages, such as C or C++ -- if you encounter a bottleneck caused by 
inefficiencies inherent in Python, you can always rewrite it in C. That's a 
good Python chant worth repeating: "You can always rewrite it in C."

Hope this helps.

-- 
Alexander Staubo     http://www.mop.no/~alex