Why don't people like lisp?

[Andrew Dalke]

Kaz Kylheku:
> The problem is that
>
>     program = compiler(character-string)
>
> is too much of a closed design. A more flexible design resembles:
>
>     abstract-syntax-tree = reader(character-string)
>
>     target-syntax-tree = translator(abstract-syntax-tree)
>
>     program = compiler(target-syntax-tree)

But you didn't see the implementation of Alex's 'compiler' function.
It looks like

def compiler(character_string):
  return compile_tree(translator(reader(character_string)))

and those intermediates are part of the publically usable
API.

> The input to a compiler should not be a character string, but
> structured data.

I think you're arguing naming preferences here.  That's fine;
Alex's point remains unchanged.  In a dynamic language like
Python, parsing a domain specific language can be done at
run-time, parsed, converted to a Python parse tree, and
compiled to Python byte codes, just like what you want and
in the form you want it.

> In Lisp, the complexity of the compiler is constant; it's a language
> builtin. The complexity of the reader depends on the complexity of the
> lexical properties of the language, and the complexity of the
> translator depends on the semantic complexity of the language.

Replace 'Lisp' with 'Python' and the result is still true.  Ditto
for 'C'.  So I'm afraid I don't understand your point.

> > The vast majority of applications has absolutely no need to tinker
> > with the language's syntax and fundamental semantics.
>
> This is a common fallacy: namely that the difficulty of doing
> something, and the consequent infrequency of doing it, constitute
> evidence for a lack of need of doing it. In fact this is nothing more
> than rationalization: ``I know I have inadequate tools, but I really
> don't need them''. It's not unlike ``I can't reach those grapes, but I
> know they are sour anyway''.

That's not Alex's argument.  Python has the ability to do exactly
what you're saying (domain language -> AST -> Python code or AST ->
compiler).  It's rarely needed (I've used it twice now in my six years
or so of Python), so why should a language cater to make that
easy at the expense of making frequent things harder?

> In Lisp, tinkering with the syntax and semantics is done even in
> trivial programs.

And that's a good thing?  That means that everyone looking at
new Lisp code needs to understand the modifications to the syntax
and semantics.  That may be appropriate for an insular organization,
but otherwise it makes it harder for others to understand any code.

> By the way, your use of ``fundamental'' suggests a misunderstanding:
> namely that some kind of destructive manipulation of the language is
> going on to change the foundations, so that existing programs are no
> longer understood or change in meaning. This is not so: rather, users
> extend the language to understand new constructs. The fundamental
> syntax and semantics stay what they are; they are the stable target
> language for the new constructs.

Alex and others have responded to this argument many times.  The
summary is that 1) in practice those who extend the language are most
often not the domain experts so the result doesn't correctly capture
the domain, 2) because it's easy to do, different groups end up with
different, incompatible domain-specific modifications, 3) rarely
is that approach better than using OOP, HOF and other approaches,
where better is defined as more flexible, easier to read, more
succinct, etc.

> This is a common viewpoint in computer science. Let me state it like
> this: ``language design is a Hard Problem that requires you to whip
> out lexical analyzers, parser constructors, complex data structures
> for symbol table management, intermediate code generation, target code
> generation, instruction selection, optimization, etc.''

Actually, language design doesn't require any of those.  They
are needed to implement a language.

Let me add that implementing a language *was* a Hard Problem,
but effectively solved in the 1970s.  The solutions are now well
known and there are a huge number of tools to simplify all
the steps in that process, books on the subject, and people with
experience in doing it.

There are still hard problems, but they are hard engineering
problems, not hard scientific ones where the theory is not
well understood.

> But when you have this:
>
>     abstract-syntax-tree = reader(character-string)
>
>     target-syntax-tree = translator(abstract-syntax-tree)
>
>     program = compiler(target-syntax-tree)
>
> you can do most of the language design work in the second step:
> translation of syntax trees into other trees. This is where macro
> programming is done, and there is a large amount of clever
> infrastructure in Lisp which makes it easy to work at this level. Lisp
> contains a domain language for language construction.

Python has all that, including the ability to turn a string into
a Python AST, manipulate that tree, and compile that tree.

It's not particularly clever; there's no real need for that.  In
general, the preference is to be clear and understandable over
being clever.  (The New Jersey approach, perhaps?)

(Though the compiler module is pretty clumsy to use.)

> > PARTICULARLY for domain-specific languages, the language's
> > designers NEED access to domain-specific competence, which typically
> > they won't have enough of for any domain that doesn't happen to be
> > what they've spent many years of their life actually DOING (just
>
> Bingo! This is where Lisp comes in; it gives the domain experts the
> power to express what they want, without requiring them to become
> compiler construction experts. This is why Lisp is used by some
> artificial intelligence researchers, biologists, linguists, musicians,
> etc.

Speaking as a representative from the biology community, the
Lisp programmers are a minority and far behind C, Fortran, Perl,
and still behind Python, Tcl, and even Ruby.

*If* the domain expert is also an expert Lisp program then
what you say is true.  It's been my experience that most domain
experts are not programmers -- most domains aren't programming.
Even in what I do, computational life sciences, most chemists
and biologists can do but a smattering of programming.  That's
why they hire people like me.  And I've found that objects and
functions are good enough to solve the problems in that domain;
from a CS point of view, it's usually pretty trivial.

                    Andrew
                    dalke at dalkescientific.com