YATOPS (Yet Another Thread on Python's Speed) (was Re: HELP! Must choose language!)

[andy]

Speed is so subjective!  A bad algorithm will make a bad program in any 
language!

I moved from using Oberon (a compiled language) to Python about 18 months ago, 
for various 'scripting' tasks at my workplace.

Ok, Oberon isn't the fastest language around - the compiler I used wasn't 
optimised, but then it does pretty well without optimisation.  I really like 
the clean-ness (to my eye) of oberon, but wanted the interactivity of an 
interpreted language.  I had exhausted the possibilities of NT batch-script, 
QBasic (yuk!) and started reading up on perl.  I was not impressed, as I have 
always preferred the 'pascal' type syntax, shunning C and its cronies.

I can't even remember how I stumbled accross Python, but I can remember 
feeling I'd 'come home' when I tried it!

The first thing I did was convert my work-in-progress directory size analyzer 
and found, to my amazement, that it ran about the same speed.  To boot, the 
os.stat function did not suffer from the same memory leaks as its oberon 
counterpart.

Recently, our company has migrated from Novell Netware 5 to Windows 2000, and 
I had to shift about 350GB of data from one server to another, in roughly 
50GB chunks.  We had tried 'robocopy' which was hopelessly slow, and I 
couldn't find any comparable products online.

I had, years ago, written an incremental directory copier in Modula-2, so I 
decided to rewrite it in Python.  Works a treat:

On a 394MB, P4/2000 windows XP pro w/s,  with 100Mb/s ethernet:

>From memory, when testing:
    copy command: roughly 600K/s
    explorer copy: roughly 500K/s
    robocopy: roughly 1000K/s

Today:
    bulkloader.py: roughly 4000-8000K/s

It uses a brute-force copy, i.e. just loads up each file and writes it to 
dest.  Good excuse to have extra memory! ;-)

I have been using this program recently to benchmark our server performance, 
as we have been experiencing performance problems (network incompatabilities 
etc.) It's been invaluable.

Python may be interpreted, and may not execute at /n/ mflops, but what it 
lacks in /raw/ oomph, it really makes up for in /efficiency/ and 
/development/ /speed/!  The stuff that needs to be fast, generally is, and 
the rest generally doesn't matter to me...

Beats batch-script senseless, and you can still read (and understand) code a 
year later (unlike perl,C/C++ etc.).

I like Python.

regards, 
-andyj