Why I love python.

[Dave Brueck]

Nick Patavalis wrote:
> On 2004-08-13, Dave Brueck <dave at pythonapocrypha.com> wrote:
>>Nobody is arguing that Python is as fast as C. But being slower does
>>not imply that Python is unsuitable for those tasks. I'd consider
>>your list to be pretty atypical of normal development (how many TCP
>>stacks and relational databases need to get written each year?),
> 
> 
> I also mentioned web-browsers, ray-tracers, circuit-simulators. I
> could add word-processor, spreadsheets, video editing programs, and
> GUI toolkits to the list. Are they still too exotic?

No - but I still don't think they reflect anywhere near a majority of 
the development that goes on.  I'm not at all saying that there aren't 
applications where performance matters, just that (1) it tends to be far 
less common than most people believe/realize (2) when it does matter, it 
usually matters in a very tiny portion of the total code, and (3) rarely 
in today's development do you need to build from scratch everything down 
to the building blocks themselves anyway (so if you're e.g. building a 
web browser, in many cases you won't even be dealing with lowest level 
data anyway - you won't be handling individual pixels but will be 
calling a library somewhere else to render an image for you - and as the 
developer of the web browser, that's just peachy)

IOW, it'd be lovely to have a blazingly fast-as-C Python, but the lack 
of fast-as-C performance is rarely the most important problem in practice.

> To cut the thread
> short, what I mean is that an application that has to do something
> like:
> 
>    for i in range(N):
>        a[i] = b[i] + c[i]
> 
> is bound to be 10 to 100 times slower than the equivalent coded in
> C. Which means that the cost of doing *computation* in Python is
> prohibitively high!

Not necessarily, and that's the point. You're making the assumption that 
  10-100 times slower is too slow. In some cases it most definitely is. 
In many cases it most definitely is not.

> Have you ever seen, say, an AVL-tree
> implementation in production Python code? Probably not. Have you ever
> seen someone implementing some sort of string-lookup algorithm in
> Python (instead of using the build-in dictionaries)? Again no. Is it
> because Python has found the "one-size-fits-all",
> "best-algorithm-ever-devised" solution?

Or is it because in 99% of the cases what is there works good enough, so 
much so that obtaining the difference is not worth the opportunity cost 
of working on something else?

If you have an overabundance of a particular resource, and you can gain 
some advantage in exchange for some of that abundance, it's nearly 
always worth the trade. Such is the case with CPU - for many, many 
programs we have oodles of excess CPU time lying around, so it's a 
worthwhile trade. And that's exactly why everybody would love a faster 
Python but most people aren't willing to invest time working on it.

I welcome any speed boost we see from people working on improving 
performance, but it's just not what's standing between me and most of my 
development goals. Heck, right now I'm writing this message, I've got my 
mail & IM messages going, a couple of code editors open, a virtual PC 
instance running my database and webserver, and I'm building web pages 
off content in the database and saving them to disk. CPU usage is 
hovering around 5%. If I were to increase the speed of all of these 
applications by a factor of 1000 I wouldn't even notice. Add a few 
features to any of them, and I would.

> Or is it because the weight of
> the language itself is such that even a suboptimal algorithm
> implemented in C will never be matched by a python implementation?

Practice has shown that, not only is this not true, but that a lot of 
times working in a higher level language is also worth it because the 
cost of discovering and implementing a better algorithm is cheaper, so 
you could end up with better performance than going to C. Or, you'd 
arrive at plenty-fast-enough sooner.

> No, but the performance difference between C and Assembly was
> *small*.

Over time, yes, but certainly not initially. I still remember how 
appallingly slow my graphics routines were in C - way too much overhead 
- while in assembly they had no problems at all.

> And at some point the C compilers became so good, that you
> couldn't beat them by hand coding something (of considerable length)
> in assembly.

That happened later, at least on PCs. The real transition happened as 
CPU speed grew so much that the difference between e.g. 4.77 MHz and 10 
MHz was boring.

> As for C++, one of its primary design-goals were "zero
> unneeded overhead"; so it *is* possible to write a C++ program that is
> as fast as a C program, if you want to do it.

It was a design goal, but (1) implementations didn't achieve it very 
well initially and (2) in the end it didn't matter that they didn't 
achieve it. More and more people migrated to C++ because the cost of 
doing so (overhead) fell steadily over time - even more quickly than the 
compilers improved.

> and I want to be sure that---by guiding
> the compiler properly---it will produce code that is as efficient as a
> well-written C program, or hand-coded assembly (or at least close to
> that).

Ugh - any time you spend guiding the compiler is time you could have 
spent actually solving a problem. I find it interesting that when 
programming in C we don't use the 'register' compiler hint anymore. Why? 
It's a combination of smarter compilers and faster computers, but either 
way its existence was awful IMO - don't distract the programmer like that.

I *love* it whenever I see that Pystone benchmarks are improving, or 
that any of the various VM implementations are making headway, and I'll 
gladly use them. But at the  same time, I have to admit that they aren't 
solving any problems I encounter on a daily basis.

> For Python to become a "primary" language, there's
> still much work to be done.

Couldn't disagree more. Yes, things like interpreters, compilers, etc. 
could use more maturity and will continue to evolve over time, but even 
_lacking_ those things it's still far enough ahead of other "primary" 
languages to make the _net_ result a huge advantage. With the evolution 
of those things the advantage will just become more pronounced.

>  But some of it has to do with
> ensuring---at the language level---that efficient environments are
> possible. 

Again, I disagree. IMO one of the benefits of higher level languages is 
that they underlying implementation technology takes care of details 
that the developer need not be concerned with.

Overall, every little performance improvement in a Python implementation 
extends the domain in which Python is a good tool for the job, and 
that's great. But AFAICT it's already a terrific fit for a massive chunk 
of real-world development, so much so that increasing its speed by, say, 
a factor of 10 isn't going to even double its domain of usability.

-Dave