Why is Python popular, while Lisp and Scheme aren't?

[Alexander Schmolck]

Pascal Costanza <costanza at web.de> writes:

> Alexander Schmolck wrote:
> > Kenny Tilton <ktilton at nyc.rr.com> writes:
> >
> 
> 
> >>Puzzled. "scale well" to me means "what happens when you try to use it for a
> >>big project" or possibly a small project with big runtime resource
> >>requirements. CL is great at either, precisely because it can do a lot.
> > "Scale well" normaly implies up, but what I mainly meant is that CL doesn't
> 
> > scale down well in a couple of ways. One factor are users: CL is a poor
> > language for people who don't have, X (say 18) months or so of time to spare
> > to learn CL (which includes pretty much everyone except people who want to
> > program professionally in CL or language afficados). This doesn't imply that
> > you can only use CL productively after learning it for X months, but unless
> > you plan to spend this overall amount, some other language (python, for
> > example) is likely to be a better choice.
> 
> Where do you get these numbers from?

The learning time? From the relevant authorities from comp.lang.lisp, of
course, in this case Erik Naggum :) 18 months is the time he expects it will
take people to become CL experts (NB: expert *in* the language *not* expert
user, which obviously takes longer):

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&safe=off&selm=3238543378011762%40naggum.no

Anyway, the number wasn't meant that seriously by me, the only thing that
matters is that it is comparatively high. The investment needed to become
productive in CL and the effect on the likelihood of people evaluating CL at
all is a more serious issue than lispers generally seem to appreciate. (Note
that by productive I mean *competitive* to alternatives you already know or
could learn instead). Clearly people will only be wiling to make such an
investment when the perceived potential gains match it. The thing is, however,
that with languages like python, you can get many of the benefits (plus
additional ones) and few of the uncertainties with much less of an investment
(read risk).


> 
> I have started to take a serious look at Lisp and Scheme in April (i.e., April
> 2002). I have started to implement a Virtual Machine for Java bytecode in
> Common Lisp in July 2002 (i.e., a just-in-time compiler that translates Java
> bytecode to Common Lisp code at load time). The core of this Virtual Machine
> executes the first few test programs since the beginning of this month
> (November 2002).
> 
> 
> So in effect, I have successfully written a piece of software in Common Lisp
> within 5 months that I wouldn't even have attempted to write in any other
> language. (And I have only spent about 8 hours per week because I have done it
> in my spare time.)

Impressive, but maybe your modesty also leads you to underestimate the
relative contributions of your own intellect and the fact that you specialize
in programming language research? 

BTW, doing something along those lines for python would also be quite
interesting (any chance that this might be your next hobby project :)?  I
guess such an effort could also be quite useful to the CL community (among
other things because python has lots of quite usable libraries and because a
successful project along those lines might attract quite a bit of attention).

> 
> For average programmers (whatever that means), I firmly believe that Common
> Lisp is among the best choices to learn quite a number of advanced concepts
> that _make_ them more professional.
> 

Yes and no. Depends on what you mean by "average". I think for those people
who are not acquainted with the advanced concepts python already offers,
python might be a better start, for two kinds of reasons:

a) conceptual: it has fewer powerful features and features of comparable
   expressiveness are often easier to use in python (e.g. keyword arguments
   and apply'ing or resizable arrays).

b) motivational: it is easier to do something small but useful with python

but I'd definitely agree that having a look at CL is most worthwhile.

> 
> It's true that end users probably don't benefit a lot from Common
> Lisp. However, I am not sure about this bit. (What you can do is to divide the
> work load among experts on the one hand who design domain-specific languages
> and end users on the other hand that use these domain-specific
> languages. Common Lisp's features are excellent for this scheme. BTW, that's
> the whole point of the field of Generative Programming.)

Well python seems quite successful as a "domain specific" language for "end
users" (in both scientific contexts and as an extension language for office
applications etc.). I haven't heard the same about CL (I guess Graham's home
page builder might qualify, but I've never seen a single application of CL for
such purposes myself).

> 
> Operations on files and directories in Common Lisp are designed to be portable
> across different operating systems. (Note that in this regard, Unix and, say,
> Windows file systems are very similar.)

Python's operations on files and directories are also portable between any OS
in practical use, as far as I can tell.

> 
> 
> You can't pick an arbitrary part of a language/library and blame it for its
> complexity. You could also state that Python doesn't make it easy to do
> conceptually simple and common things and then name macro programming as an
> example. Macro programming is in fact conceptually simple and common in Lisp!
> 

Conceptually simple? To a computer scientist, yes, but I guess lambda calculus
is also conceptually simple. Common? Well, everyone I read on macros and CL
always stressed that they should be only used most sparingly (Graham, a big
fan of macros, says he'd only trust a few people enough to actually write
some).

> 
> Each language has its strengths and weaknesses in different areas. I don't

Yes, but some languages are crap and others aren't.  Also we were not talking
about strengths and weaknesses in general but accessibility (which certainly
can be traded off against other desirable properties).

> know a single language that's equally strong in all areas. (So I think that
> neither your statement nor my statement made up on the spot about macro
> programming in Python make any sense.)
> 

Well, I beg to differ as far my statement is concerned :) File operations and
directory operations are fairly basic and ubiquitous -- if they are awkward
for some reason it does affect the accessibility of a language (unless it
isn't designed for such operations because, say it is meant for mobile
telephones).

> 
> > 2. try to write something in 1-100 lines in CL that couldn't easier be
> >    written in e.g. python.
> 
> Hello World in Common Lisp:
> 
>  > "Hello, World!"
> "Hello, World!"
> 
> I hope you get the joke. ;-)


Afraid not. Would it help if I added "something practically useful"?

> 
> > 3. just think of how many ways there are to express quite similar
> >    conditional or looping constructs.
> 
> How do you measure readability?
> 

Good question. I really think ergonomical questions like this would merit more
empirical research. Unfortunately, language designers don't generally seem to
be that interested into psychological research or usability studies (python's
predecessor ABC, if I'm not mistaken, has received some attention along those
lines, however).

But here is what an highly experiences Lisper (Peter Norvig) has to say on the
matter:

"Python seems to be easier to read than Lisp for someone with no experience in
either language. The Python code I developed looks much more like the
(independently developed) pseudo-code in the book than does the Lisp
code. This is important, because some students were complaining that they had
a hard time seeing how the pseudo-code in the book mapped into the online Lisp
code (even though it seemed obvious to Lisp programmers)."


> Meiner Meinung nach hängt das im wesentlichen immer davon ab, welche Sprache
> man zuerst gelernt hat, oder welche man am häufigsten benutzt.

[Pascal stresses (prior) exposure]

I don't think so. There are limits on what our minds can do well and fast,
irrespective of how much practice we receive. Show me someone who'll read
machine code faster than python. I suspect that prior experience is more
important for shaping irrational predilections (such as C's stupid braces or
static typing). I and others balked at python's use of whitespace on first
exposure, but found it much more readable within very short time.

> 
> 
> If you don't understand this, would you consider German to be less readable
> than English? (Run that sentence through http://babelfish.altavista.com to get
> my point. ;)

Nicht notwendig ;)

> 
> 
> > 4. try e.g. modifying constant lists or writing a LOOP with undefined
> >    behavior, or modifying the (string) keys in a hash-table.
> >
> 
> 
> In any language I know, beginners make similar mistakes.
> 

There are obviously pitfalls and undefined behaviors in every language, but
they are most certainly *not* all equal in this regard. Python is specifically
designed with the aim in mind to make it *difficult* for people to shoot
themselves in the foot, even if it's at the expense of conciseness or
expressiveness.

For example you can't assign in a statement:

while c = f.readline():
   ...

is illegal in python. Similarly, strings are immutable and mutable types like
lists don't come with __hash__ methods, so they can't be used as dict
(i.e. hashtable) keys.


> >>and then one just keeps on learning more and more cool stuff because it is
> >>so rich.
> > Well, in python you can also learn new cool stuff till you reached a fairly
> 
> > high level (e.g. generators and metaclasses), but most of the code you write
> > will still be quite readable to less clued programmers, because simple things
> > are invariably expressed in pretty much the same way. So while a newbie might
> > have problems groking my generator code, most other things should still be
> > ok. I want to see the CL newbie that understands e.g. the series, dolist, loop
> > and mapcar etc. version of the same algorithm.
> 
> Common Lisp is not designed for newbies but for experts. 

Well the funny thing is, python is designed for newbies *and* experts and it
works remarkably well.

from www.norvig.com:

"Python supports all of Lisp's essential features except macros, and you don't
miss macros all that much because it does have eval, and operator overloading,
and regular expression parsing, so you can create custom languages that way."

[...]

"Nowhere did I feel that Python was missing something that made it hard to
implement something I had done in Lisp."

And not only is that from somebody who wrote one of the best lisp books, the
last statement also refers to porting his AI code, an area were CL is supposed
to be especially strong.


> IMO, newbies should aim at becoming experts. 
I disagree. Thankfully most people don't waste valuable lifetime in a quest to
become expert programmers (the sorry state of the computing industry
invariably means that in the process of becoming an expert you have to
memorize copious amounts of trash). I wouldn't really want to change
that. There are plenty of legitimate and important uses of programming by
people who neither have the resources nor care to become expert programmers.

Naturally, professional programmers are a different matter and I'd really like
to see more people learn lisp (rather than say, C++).

> See http://www.paulgraham.com/avg.html for some insights related to this
> point.

Sure, I know Graham's essay and I think he makes some good points, but I find
his secret-weapon-lisp argument quite unconvincing (how many success stories
exist for all sorts of (sometimes quite demanding) projects done in crap
languages like C? Do you know any other examples except Graham of people whom
CL made rich?). Either people are really good at keeping their secret weapons
secret, or lispers overestimate the productivity gain of CL over alternatives.

alex

P.S. I really got to do some work now, so I might not be able to reply till
sometime over the weekend.