[Python-ideas] Python as meta-language
talin at acm.org
Sun Dec 24 08:55:41 CET 2006
One of my long-standing interests is in mini-languages, particularly
declarative languages. I'm always looking out for examples where a
declarative language is used to represent some idea or concept that is
not easily written in an imperative language. Examples are the behaviors
of particle systems, kinematic constraints, formalized grammars, logical
inferencing systems, query languages, and so on. In other words, you are
using a language to describe a complex set of relationships, but you
aren't giving specific commands to execute in a specific order.
Often such mini-languages are implemented by writing a custom parser.
However, often this is not necessary if the underlying language (such as
Python) is flexible enough. Python's ability to declare complex
structures as literals, combined with its ability to overload operators,
means that one can often embed the mini-language within the Python
syntax itself, and use the Python compiler as the parser for your mini
language. It also allows you a convenient means to "escape" back into
the procedural world when needed.
Examples of the kind of things I am talking about include the SConstruct
file format from SCONS and the SQLBuilder syntax from SQLObject. And
although it's not directly related to Python, JSON has a lot of the same
ideas - that is, using a scripting language source code as an efficient
representation of complex data structures. And these are just a few of
the many examples out there.
What I'd be interested in doing, in this python-ideas list, is
brainstorming some ideas for how we can improve Python's ability to
'host' other kinds of mini-languages within the Python syntax. We can
start perhaps by examining some of the use cases I listed in the first
paragraph - particle systems, etc - and see how one would want to
represent those kinds of semantic structures within Python.
Of course, there are some languages (Lisp and Dylan come to mind), which
are even more flexible in this regard - the languages can be 'morphed'
out of all recognition to the original syntax. (For example, in Dylan, a
'macro' was not a simple textual substitution as in C, but in fact added
new production rules to the parser.) In I'm in no way advocating such a
course. (Well, at least not at this moment :) ) I certainly recognize
that there is a danger in making a language too 'plastic', in that it
can easily be obfuscated with too much cleverness and lose it's
identity. So I'm more interested in ideas that are subtle yet powerful.
I'm sure that there are lots of approaches to this general concept. I'm
going to throw out a couple of ideas, but I am going to post them as
separate replies to this email, and not right away - the reason is, I
don't want this thread to be taken over by the discussion / criticism of
those specific ideas, when I'm more interested in brainstorming the
More information about the Python-ideas