Python dialect that compiles into python
Many features on this list propose different syntax to python, producing different python "dialects" that can statically be transformed to python : - a,b += f(x) → _t = f(x); a += _t; b += _t; (augmented assignement unpacking) - a = 2x + 1 → a = 2*x + 1 (juxtaposition is product) - f(*, x, y) → f(x=x, y=y) (simplekwargs) - DSL specific language - all def become @partially def - etc... Using a modified version of ast, it is relatively easy to modifiy the syntax tree of a program to produce another program. So one could compile the "python dialect" into regular python. The last example with partially for example doesn't even need new syntax. Those solutions that are too specific would then be provided as a module on pip that has a common interface for "compiling" : $ cat test.dialect.py<http://test.dialect.py> #! dialect: juxtaposition a = 2x + 1 $ python -m compile test.dialect.py<http://test.dialect.py> $ cat test.py #! compiled with dialect juxtaposition a = 2x + 1 The generated file should also be read only if the filesystem provides the option. In the web world, it's very common to compile into html, css or js. One of the reason was that the web must be veeeery generic and will not try to fit everyone needs. - less compiles scss into css - coffeescript into js - source map provides a standard way to map each line of the new file into lines of the old files (useful for exceptions !) One useful feature of those compilers is the --watch Option that allows to avoid to launch the compilation manually. Of course, in the js world, the syntax was improved in the end after a long maturation of the compiling and not compiling libraries. In the java world, languages such as Scala compile into bytecode, that's another idea. If a standard module like "compile" is written, users can write their own module that will automatically be read by "compile" (for example, pip install compile_juxtaposition would allow the juxtaposition dialect). Compile doesn't even have to be on the standard python, it can be a lib. One could write a module using multiple dialect like dialect: juxtaposition, simplekwargs The order would be technically important but functionally non important. Actually, I might start to write this lib, that looks fun.
Many features on this list propose different syntax to python, producing
For my specific suggestion (simplekwargs), it's really not worth doing at all if it's not a part of standard Python in my opinion. The reason is tooling: you wouldn't get tools like Jedi and PyCharm to read these files and interpret them correctly. That being said, it could absolutely be a nice way to prototype things or for very edge cases. I'd recommend looking into parso (the AST lib that underlies Jedi). I used it for the analysis tool I wrote for simplekwargs and I've also used it to write my mutation tester mutmut. It has several advantages: - it's a roundtrip AST so you wouldn't lose any formatting or comments etc - it has an error recovery mode where parse errors are isolated in the AST as error nodes. You could probably use this to just enumerate error nodes and doing .get_code() on them and then have all your special parsing in there. - it is very liberal in what it accepts putting into an AST that it renders out to source again so it's quite easy to hack something together that works / Anders
On Fri, Sep 07, 2018 at 11:57:50AM +0000, Robert Vanden Eynde wrote:
[...]
[...]
Actually, I might start to write this lib, that looks fun.
I encourage you to do so! It would be great for non-C coders to be able to prototype proposed syntax changes to get a feel for what works and what doesn't. There are already a few joke Python transpilers around, such as "Like, Python": https://jon.how/likepython/ but I think this is a promising technique that could be used more to keep the core Python language simple while not *entirely* closing the door to people using domain-specific (or project-specific) syntax. -- Steve
[Steven D'Aprano]
It would be great for non-C coders to be able to prototype proposed syntax changes to get a feel for what works and what doesn't.
I think it would be great in general for the community to be able to try out ideas and mull things over. If there was something like a Python Feature Index (PyFI) and you could install mods to the language, it would allow people to try out ideas before rejecting them or incorporating them into the language (or putting them on hold until someone suggests a better implementation). I could even see features that never make it into the language, but stick around PyFI and get regular maintenance because: A) they're controversial changes that some love and some hate B) they make things easier in some domain but otherwise don't warrant adoption It would have to be made clear from the start that Python can't guarantee backward compatibility with any mods, which should prevent excessive fragmentation (if you want your code to be portable, don't use mod). On Fri, Sep 7, 2018 at 7:30 PM Steven D'Aprano <steve@pearwood.info> wrote:
Le 11/09/2018 à 02:15, Abe Dillon a écrit :
That would be an almost-Python to Python transpiler, I love it! It would surely help a lot to explain and try out new ideas, as well as for domain specific needs. And having an index of those features could help a lot. It would surely also help a lot with backward compatibility of new functionalities. Someone who would want to use a Python 3.9 functionality in 3.8 (whatever his reasons) could use a shim from the index. Such shim wouldn't have to be as optimal or fast as the version that would be used in 3.9, but it could be functionally equivalent to it. I'm just a bit afraid of the popularity that some of these experiments could get (like `from __future__ import braces` -> No problem!) and the pressure that could be made upon core devs to push the most popular changes into Python (popularity not being equivalent to sanity for the language and its future. There are many devs that want to code in one language the same way they do in others, which is often wrong, or at least not optimal).
Op di 11 sep. 2018 om 10:42 schreef Brice Parent <contact@brice.xyz>:
For what it's worth, the Ocaml community has something like that: Campl5 https://camlp5.github.io/doc/html/ Despite the name "preprocessor" this actually communicates with the Ocaml compiler proper through an AST. So you get proper source code location, etc. I think you could actually already hack something like this in Python today, by creating custom import hooks, which then run your own compile step on a file, which produces an AST, which is then passed to compile(). However keeping your custom Python++ parser in sync with Python is probably a pain.
I fully expect the core devs to resist such pressure, especially for the braces ;-) Stephan
On 09/07/2018 04:57 AM, Robert Vanden Eynde wrote:
Actually, I might start to write this lib, that looks fun.
You should also check out MacroPy: https://pypi.org/project/MacroPy/ Although I freely admit I don't know if does what you are talking about. -- ~Ethan~
Many features on this list propose different syntax to python, producing
For my specific suggestion (simplekwargs), it's really not worth doing at all if it's not a part of standard Python in my opinion. The reason is tooling: you wouldn't get tools like Jedi and PyCharm to read these files and interpret them correctly. That being said, it could absolutely be a nice way to prototype things or for very edge cases. I'd recommend looking into parso (the AST lib that underlies Jedi). I used it for the analysis tool I wrote for simplekwargs and I've also used it to write my mutation tester mutmut. It has several advantages: - it's a roundtrip AST so you wouldn't lose any formatting or comments etc - it has an error recovery mode where parse errors are isolated in the AST as error nodes. You could probably use this to just enumerate error nodes and doing .get_code() on them and then have all your special parsing in there. - it is very liberal in what it accepts putting into an AST that it renders out to source again so it's quite easy to hack something together that works / Anders
On Fri, Sep 07, 2018 at 11:57:50AM +0000, Robert Vanden Eynde wrote:
[...]
[...]
Actually, I might start to write this lib, that looks fun.
I encourage you to do so! It would be great for non-C coders to be able to prototype proposed syntax changes to get a feel for what works and what doesn't. There are already a few joke Python transpilers around, such as "Like, Python": https://jon.how/likepython/ but I think this is a promising technique that could be used more to keep the core Python language simple while not *entirely* closing the door to people using domain-specific (or project-specific) syntax. -- Steve
[Steven D'Aprano]
It would be great for non-C coders to be able to prototype proposed syntax changes to get a feel for what works and what doesn't.
I think it would be great in general for the community to be able to try out ideas and mull things over. If there was something like a Python Feature Index (PyFI) and you could install mods to the language, it would allow people to try out ideas before rejecting them or incorporating them into the language (or putting them on hold until someone suggests a better implementation). I could even see features that never make it into the language, but stick around PyFI and get regular maintenance because: A) they're controversial changes that some love and some hate B) they make things easier in some domain but otherwise don't warrant adoption It would have to be made clear from the start that Python can't guarantee backward compatibility with any mods, which should prevent excessive fragmentation (if you want your code to be portable, don't use mod). On Fri, Sep 7, 2018 at 7:30 PM Steven D'Aprano <steve@pearwood.info> wrote:
Le 11/09/2018 à 02:15, Abe Dillon a écrit :
That would be an almost-Python to Python transpiler, I love it! It would surely help a lot to explain and try out new ideas, as well as for domain specific needs. And having an index of those features could help a lot. It would surely also help a lot with backward compatibility of new functionalities. Someone who would want to use a Python 3.9 functionality in 3.8 (whatever his reasons) could use a shim from the index. Such shim wouldn't have to be as optimal or fast as the version that would be used in 3.9, but it could be functionally equivalent to it. I'm just a bit afraid of the popularity that some of these experiments could get (like `from __future__ import braces` -> No problem!) and the pressure that could be made upon core devs to push the most popular changes into Python (popularity not being equivalent to sanity for the language and its future. There are many devs that want to code in one language the same way they do in others, which is often wrong, or at least not optimal).
Op di 11 sep. 2018 om 10:42 schreef Brice Parent <contact@brice.xyz>:
For what it's worth, the Ocaml community has something like that: Campl5 https://camlp5.github.io/doc/html/ Despite the name "preprocessor" this actually communicates with the Ocaml compiler proper through an AST. So you get proper source code location, etc. I think you could actually already hack something like this in Python today, by creating custom import hooks, which then run your own compile step on a file, which produces an AST, which is then passed to compile(). However keeping your custom Python++ parser in sync with Python is probably a pain.
I fully expect the core devs to resist such pressure, especially for the braces ;-) Stephan
On 09/07/2018 04:57 AM, Robert Vanden Eynde wrote:
Actually, I might start to write this lib, that looks fun.
You should also check out MacroPy: https://pypi.org/project/MacroPy/ Although I freely admit I don't know if does what you are talking about. -- ~Ethan~
participants (7)
-
Abe Dillon -
Anders Hovmöller -
Brice Parent -
Ethan Furman -
Robert Vanden Eynde -
Stephan Houben -
Steven D'Aprano