Hello André, On Sun, 29 Nov 2020 13:07:48 -0400 André Roberge <andre.roberge@gmail.com> wrote: ]
Thanks for posting this proposal. It should be pretty clear that's the best process which should be followed.
Thank you.
It's also should be pretty clear that (almost) nobody follows it. If anything, lack of responses by the substance of the matter proposed, i.e. the *process*, not technical means to achieve it, is very telling.
I had taken the lack of response as indicative that my idea was simply "wrong".
I pray for that to not be the case. People who *do* stuff shouldn't be discouraged so easily. Let doubts fall on people who get a random idea and 5 minutes later it's on the list, without any attempt of research or something on their side. []
Aside: some fairly complex transformations can be implemented using codecs; see https://github.com/dropbox/pyxl for example.
Alternatively, a program can be parsed, modified and dumped again as a stream of characters, just to immediately be parsed by the Python interpreter again. In either case, it's a hack, and should be rejected as a viable approach to a problem domain.
I'd say that your link proves my point: that's a roundabout and complex way, but of course, whole companies ("dropbox" in the link above) can afford themselves to wade thru it, for the lack of a better alternative.
Now you say that the second choice is "import hooks". But import hooks in Python is its own kingdom of "wonderland". Import hooks aren't intended to "allow to experiment with syntax", they are intended to do anything what's possible with module loading (and many things that aren't possible, I would add). So, they have rather complex, non-intuitive API, thru which anyone would need to wade, neck-deep, before implementing something useful.
In my original email, I linked to a project I created ("Ideas": https://aroberge.github.io/ideas/docs/html/) which meant to simplify this as much as possible.
In other words, both choices you list aren't really viable to experiment with syntax/semantics.
I respectfully disagree, based on my own experimentation, documented in "Ideas" mentioned previously. For example, I implemented module level constants using this approach.
I know. I also experimented with it. But that didn't leave me with a feeling of achievement. If anything, it left me with a feeling of powerlessness, which I tried to convey in my reply. When I think of all the people who need to wade thru it again and again, and the whole thing is purposely made hard/confusing for people to use (PEP 511 rejection, again), and how many gave up on that way - that doesn't make me feel good. I'm well aware that many people trod that way nonetheless. But no, it's not many, it's "a few", and that's the problem. For example, your "Ideas" project sits in my browser tab nearby to https://jon.how/likepython/ , which likely means I got links of them together. That's from 2010, and I don't think it's first of its kind or anything. And let me know if it helped you with your task, or you had to retrace the path from scratch. Anyway, I don't consider "I did that" to be good criteria. "Every 2nd Python user can do that easily" would be. (Ok, let's be realistic - every 10th).
They *almost* work ... except for some corner cases due to the fact that I am using module objects created using Python's type() as I have not figured out (yet?) how to create a module object with a custom dict.
So, I postponed replying, because even by reading "part 1" in your blog I got an idea that we were working of very close things. So, I was anxious to get my proposal on the "strict mode" out, which includes https://github.com/pfalcon/python-strict-mode , a strict mode prototype for CPython - to have a bit of the "clean room" (after-the-fact) comparison. So, I just finished reading https://aroberge.blogspot.com/2020/03/true-constants-in-python-part-2-and.ht... , and yeah, we were following almost the same way. I also implemented that "custom module object" you write about. Except it's not custom at all. It's a generic "read-only proxy" object I write about in https://mail.python.org/archives/list/python-ideas@python.org/message/7TRMSB... In my Pycopy, it lives in sys.roproxy. In the CPython prototype it's coded approximately at https://github.com/pfalcon/python-strict-mode/blob/master/strict.py#L52 . I wrote that code as a part of personal "advent of code" hacking of the winter holiday season of 2019/2020. So, if you have cookies for your "contest" winners, count me in the line ;-). []
What's needed is higher-level API specifically for the usecase of letting Python source of modules to run thru custom tokenizer/parser/bytecode compiler. The best known (non-adhoc) approach to that is PEP511: https://www.python.org/dev/peps/pep-0511/.
I believe that the intention of PEP-511 was to optimize the code generated, and not to experiment with different syntax. I *think* that my ideas project is essentially a superset of what is described in pep-511, but with a completely different goal.
Glad to hear you (seem) to promote your "Ideas" as *the* solution to the issue. We need people making bets like that. But how does it compare to other folks' stuff? You say you implemented Decimal literals? And this guy implement Roman literals: https://github.com/isidentical-archive/pepgrave (of a 1st-april PEP). Whose approach is better? And he writes "Re-write of my old project PEPAllow with a complete new approach", and "new" doesn't always mean "better", so his old approach should be in the competition too. And I guess that's the only way to do it "right" - take stuff from different people (including not yet written stuff) and "throw it to arena" to find the best.
Best,
André
--------
This turned out to be a long intro. As I said, I fully agree with you, that changes to Python should be prototyped in Python. And there's no better way to "agree" than actually dogfood this approach to oneself.
Yes. One additional comment, point of emphasis about my original proposal.
Take PEP 505 (None-aware operators) written 5 years ago. It is still deferred.
Imagine that someone wants to experiment with a custom implementation. Using the approach I mentioned, it should be possible to do so.
Suppose that the same person wishes to also experiment with block-level scoping. By using the approach I mentioned in my original message, it would be possible to "stack" transformations and experiment with code that support either None aware operators and/or block-level scoping. Different syntax variants could be implemented separately and, provided they do not create conflicts, could be tested together so that people could get a feel as to whether or not some proposed change should be included in Python.
Absolutely. As I said, that's the right approach. Block-level vars can be implemented purely by AST rewriting for example. Now people interested in that approach should get together, take each own project as a shield and sword, find out whose tools are the best, everyone else to abandon their own "projects of love and sweat", and then with common tools get to cooperate on achieving that aim, instead of duplicating each other's efforts, multiplying incompatible solutions. If that sounds almost impossible, then certainly it is. So, let's get to work! [] -- Best regards, Paul mailto:pmiscml@gmail.com