Python Performance

[aaron_watters at my-deja.com]

These points are all valid, but they can be deceptive.
I add some comments
(not meant to negate the valid observations):

In article <al81zdxkktf.fsf at sirppi.helsinki.fi>,
  Markus Stenberg <mstenber at cc.Helsinki.FI> wrote:
> Even in websites' case, I would worry; Python doesn't use multiCPU
very
> well, and with single CPU, it is fairly easy to saturate one with
> pure-Python code, especially if it is badly designed to boot.

Bad design aside.  If you have many CPU's for a web site you
might consider using plain old CGI scripts, since python would then
use all the cpu's.  In a single (multithreaded) process python will
only use one (thanks to the global interpreter lock).  Also, I
strongly feel CGI is more stable and reliable than other methods.

> Using a Gadfly database, _with_ pre-compiled queries: (very simple
> one-liners from table with one entry)
> select: 21.4182/sec [3.64s] (46.689ms/call)
> update: 14.3209/sec [3.98s] (69.828ms/call)

Note please that gadfly queries have a certain amount of fixed overhead.
You *might* find that queries over a table with one entry are not
much faster than queries over tables with 1000 entries, or maybe even
tables with 10000 entries that have appropriate indices.

However each gadfly query requires 10's of python function
calls regardless of how simple the query is.  But more complex queries
over more data don't necessarily require many more python function
calls (this was one of the design goals for the engine).

Also, I hope you have added the kjbuckets builtin...

On second thought 21 queries per second ain't too bad...
What are you comparing it to?  Updates are naturally slower
since they involve writes to a log file for recovery purposes.

> .. and so on. Even more distressing results lie in the speed of say,
> implementing protocols in Python (when we're talking _nearly_ realtime
> action).

Be careful here.  Protocol speed can be effected by a lot of stuff.
Some of it can be python and some of it can be other system overhead.
If a profiler or unix "top" says it's the python process and it's
implemented well, then you've got a point.  But until I see the
code I personally would withhold assignment of blame.

   -- Aaron Watters

===

less is more


Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.