Domain Specification
Many fields develop their own specific short-hand notation and vocabulary for communicating the complex yet recurring ideas within that field easily. In many cases, this is a modification of an existing language or, in some cases, many different languages. This idea is a bit half-baked, but what if we could define a meta language that allows people to describe how to alter Python into some domain-specific language. Code written in the meta-language would be called a "domain specification" and it would describe new reserved-words and syntax for that DSL. It might even release some reserved words from reservation. This might provide an interesting way to try new syntax proposals. I don't know how something written in a DSL would interoperate with regular Python, they may have to use separate interpreters and pass messages between the two. I also don't know how you would link a script to a domain specification. Maybe this idea is less than half-baked. 1/4th baked? That way someone could write a version of Python where the assignment operator is "→" or "->" or "(╯°□°)╯︵ ┻━┻".
On Tue, Nov 12, 2019 at 10:52 AM Abe Dillon <abedillon@gmail.com> wrote:
I'd recommend exploring two paths: 1) MacroPy 2) Preprocessors/compilers MacroPy is an insanely powerful tool for messing around with Python. You may well be able to do what you want that way. If you can't, though, what I'd recommend is a parser that reads your modified script and spits out a valid Python program - you can either think of that as a preprocessor (if it makes only a few small tweaks here and there), or as a compiler that emits Python code (if it makes extensive changes). If you want to generalize, as you're saying here, then you could have a generic preprocessor/compiler that takes a domain specification and a source file, and spits out the corresponding Python code. Since Python code is all text, it's something you can very easily manipulate with Python itself. ChrisA
On Nov 11, 2019, at 15:58, Chris Angelico <rosuav@gmail.com> wrote:
It’s worth noting that you can very easily put a preprocessor or transpiler in an import hook. If you want to make import spam look for spam.pb flies as well as spam.py and spam.so, and handle them by calling your preprocessor and then executing the result as normal Python code, no problem. Or if you want to require from pb import spam, that’s not much harder. Or if you want to process all .py files and transpire the stuff between pairs of magic comments from your language to normal Python before letting it execute, easy. Plus, you don’t have to do it as text. You can hook things as raw bytes (if you want to override Python’s standard coding declaration or UTF8 default), or as a stream of tokens, or as an AST tree, or as compiled bytecode. Or two or more in a single hook—re.sub some not-valid-as-Python text into valid-but-useless placeholder text, then compile normally up AST, then handle the placeholder in an AST transform, then compile the rest of the way. This is usually a lot easier than modifying the Python parser or writing your own Python-plus parser, at least in the early exploratory phase. But definitely try MacroPy first, as Chris said. It does most of the heavy lifting of building and installing an import hook that runs your macros as AST transforms, so you only have to focus on what the macros should look like and do. It can’t always do everything you need, but when it can, it’s amazing.
On Tue, Nov 12, 2019 at 10:52 AM Abe Dillon <abedillon@gmail.com> wrote:
I'd recommend exploring two paths: 1) MacroPy 2) Preprocessors/compilers MacroPy is an insanely powerful tool for messing around with Python. You may well be able to do what you want that way. If you can't, though, what I'd recommend is a parser that reads your modified script and spits out a valid Python program - you can either think of that as a preprocessor (if it makes only a few small tweaks here and there), or as a compiler that emits Python code (if it makes extensive changes). If you want to generalize, as you're saying here, then you could have a generic preprocessor/compiler that takes a domain specification and a source file, and spits out the corresponding Python code. Since Python code is all text, it's something you can very easily manipulate with Python itself. ChrisA
On Nov 11, 2019, at 15:58, Chris Angelico <rosuav@gmail.com> wrote:
It’s worth noting that you can very easily put a preprocessor or transpiler in an import hook. If you want to make import spam look for spam.pb flies as well as spam.py and spam.so, and handle them by calling your preprocessor and then executing the result as normal Python code, no problem. Or if you want to require from pb import spam, that’s not much harder. Or if you want to process all .py files and transpire the stuff between pairs of magic comments from your language to normal Python before letting it execute, easy. Plus, you don’t have to do it as text. You can hook things as raw bytes (if you want to override Python’s standard coding declaration or UTF8 default), or as a stream of tokens, or as an AST tree, or as compiled bytecode. Or two or more in a single hook—re.sub some not-valid-as-Python text into valid-but-useless placeholder text, then compile normally up AST, then handle the placeholder in an AST transform, then compile the rest of the way. This is usually a lot easier than modifying the Python parser or writing your own Python-plus parser, at least in the early exploratory phase. But definitely try MacroPy first, as Chris said. It does most of the heavy lifting of building and installing an import hook that runs your macros as AST transforms, so you only have to focus on what the macros should look like and do. It can’t always do everything you need, but when it can, it’s amazing.
participants (3)
-
Abe Dillon -
Andrew Barnert -
Chris Angelico