Python vs. Lisp -- please explain

[Alexander Schmolck]

63q2o4i02 at sneakemail.com writes:

> Hi, I've been thinking about Python vs. Lisp.  I've been learning
> Python the past few months and like it very much.  A few years ago I
> had an AI class where we had to use Lisp, and I absolutely hated it,
> having learned C++ a few years prior.  They didn't teach Lisp at all
> and instead expected us to learn on our own.  I wasn't aware I had to
> uproot my thought process to "get" it and wound up feeling like a
> moron.
> 
> In learning Python I've read more about Lisp than when I was actually
> trying to learn it, and it seems that the two languages have lots of
> similarities:
> 
> http://www.norvig.com/python-lisp.html
> 
> I'm wondering if someone can explain to me please what it is about
> Python that is so different from Lisp that it can't be compiled into
> something as fast as compiled Lisp?  

Nothing. Given a sufficiently smart implementation any language can be as fast
as any other -- it might just be a billion times harder to write that
implementation for language A than for language B.

> From this above website and others, I've learned that compiled Lisp can be
> nearly as fast as C/C++, so I don't understand why Python can't also
> eventually be as efficient? Is there some *specific* basic reason it's
> tough? Or is it that this type of problem in general is tough, and Lisp has
> 40+ years vs Python's ~15 years?

I think if you're looking for one single reason, it is presumably that (IIRC)
python was designed on the erronous assumption that dynamically typed
languages have to be slow (and are unsuitable for real applications anyway)
wheras common lisp wasn't. Furthermore the people involved in common lisp were
much more knowledgeable and experienced in things like compiler design and had
a long history of similar languages and various implementations to build upon.

As common lisp and scheme demonstrate you can have high level of dynamism (and
in a number of things both are more dynamic than python) and still get very
good performance (in some cases close to or better than C). But both these
languages have been designed with compiler writers and the generation of fast
code in mind, so they made design decisions to ease writing fast lisp
compilers and programs.

For example:

- python classes (and to some extent modules) are essentially dictionaries
  that you can modify and customize more or less at will at run-time and that
  behave interchangeably in many respects. I'm sure that presents several
  optimization headaches.

  By contrast if the common lisp compiler sees the symbol CL:LIST (usually
  written just LIST, because the CL package is imported by default) it can
  safely assume that it refers to the builtin LIST function, because you're
  not allowed to rebind the function value of functions in the CL package.
  Python can assume no such thing if it comes across ``list`` -- for all it
  knows it might as well be the number 42. Also the package and class system
  are completely separate and although common lisp's OO system is rather more
  powerful than python's it has been designed to be implementable efficiently.

- in python almost everything has to happen at run-time, whereas in common
  lisp you can do things at compile time, load time or run-time e.g:

    - common lisp has a mechanism for making compiler declarations (so you can
      tell the compiler to inline a function, or the type of a variable, or to
      optimize something for speed and not for space etc.)

    - common lisp has macros (proper ones, not C style) which allow you to build
      efficient abstractions

    - common lisp has compiler macros. This sort of means that you can write your
      own compiler optimizations for your functions (i.e. if you know that your
      expensive FOO function is indempotent you could arrange for all calls of the
      form (FOO (FOO A)) to be replaced with simply A, in a similar way as an
      optimizing compiler might replace (a+b+c+d)*0 with 0).

What's far more interesting to me, however, is that I think there a good
reasons to suspect python's slowness is more of a feature than a flaw: I'd not
be suprised if on the whole it greatly increases programmer productivity and
results in clearer and more uniform code.

If you know the language to be dog slow any way, you're much less likely to
waste your time (and that of future maintainers) on the pointless
microoptimizations that geeks so love. Also, since only builtins have
reasonable performance there's added motiviation to become very familiar with
the available builtins (and standard libarary) and far less temptation to roll
one's own version of say dict.setdefault (even if it it sucks). The fact that
non-standard library code is inherently somewhat inferior (because it will
either be written in python and slow or written in C and a pain to install)
adds further incentive to attempt community wide standardization.

I think it's not unreasonable to speculate that all this decreases production,
maintenance and reuse costs of python code considerably, so much in fact that
python's very slowness represents part of its competetive edge over languages
that are in some ways better engineered and more capable.

So ironically, some share of python's success might actually be due to
ignorance on Guido's part (of course python in most respects is a marvellously
well designed language that to my mind places Guido at least on par with the
designers of any contemporary language; I'm just intrigued by the possiblity
that had he known as much about performance issues in very dynamic languages
as say some of the lisp and self people, python might have turned out to be a
faster albeit less popular and productive language).

'as