looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[Carl Banks]

On Jan 6, 9:32 am, mark <mark.fi... at googlemail.com> wrote:
> I want to implement a internal DSL in Python. I would like the syntax
> as human readable as possible. This means no disturbing '.;()\'
> characters. I like to have the power of the hosting language as well.
> Thats why I want to build it as an internal DSL and NOT as a external
> DSL.
>
> I want the DSL as human readable as possible:
>
> open_browser
>
> navigate_to 'www.openstreetmap.org'website
>
> search 'Von-Gumppenberg-Strasse, Schmiechen'
>
> verify search_result
>
> zoom in

In the Python grammar, there are no non-trivial situations where two
expressions can be separated by whitespace and not punctuation.  (The
trivial exception is string concatentation.)

String constants like 'www.openstreetmap.org' and identifiers like
open_browser are expressions, and if you try to separate them with
whitespace you get a syntax error.

So you can't make an internal DSL like this that uses Python's built-
in grammar.  You'd have to hack the parser or settle for an external
preprocessor.



> Martin Fowler recommends "Method Chaining" to build internal DSLs:
>
>  Browser("http://www.openstreetmap.org/") \
>         .search("Von-Gumppenberg-Strasse, Schmiechen") \
>         .zoom_in()
>  <<<
>
> I guess that it is possible to argue that this means the same.
> Nevertheless I do not like all the parentheses and punctuation
> necessary to satisfy the Python interpreter.
>
> The reason why I need this is that I want to have non technical people
> review the files written in the DSL.
>
> I already know that there are parser frameworks available but I want
> to build it as internal DSL in Python (Yes, I know ANTLR, Ply, and
> whatnot).
>
> How would one approach this in Python? Do I need to build a custom
> loader which compiles *.dsl files to *.pyc files? Is it possible to
> switch between the custom DSL and the standard Python interpreter?

I don't know specifically what you mean my "custom loader", or
"switching between the custom DSL and the standard Python
interpreter".

However, the gist of it seems to be that you want to be able to write
files in your DSL that can be imported just like a regular Python
module.  Yes, that can be done.

See PEP 302, Import Hooks:

http://www.python.org/dev/peps/pep-0302/

Python's standard importer looks for files with *.py, *.pyc, *.pyd, or
*.so extensions.  You could write an importer that looks for *.dsl
files, and, instead of loading it as a Python file, invokes your DSL
parser.


Carl Banks