Another thing I had not considered is the effect this would have on the other implementations. I have no clue as to how difficult all this would be to accomplish in the DLR, or the JVM, or pypy, or others. I have no doubt that they could find a way, but for all I know it would put an undue burden on them. Is that a criteria for Python feature consideration, the impact on other implementations of adding the feature?<div>
<br></div><div>-eric<br><br><div class="gmail_quote">On Tue, Mar 29, 2011 at 12:58 PM, Eric Snow <span dir="ltr"><<a href="mailto:ericsnowcurrently@gmail.com">ericsnowcurrently@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
I appreciate your candor. You are right that the bytecode would have to live elsewhere, probably memory. Or you would have to handle the .pyc generation dynamically. namedtuples do dynamic code generation and execution, but not on the same scale as def-from would cause.<div>
<br></div><div>Unless I misunderstand (not unlikely), it seems like your objection is that the code for the builder in a def-from would not necessarily be around yet to build the bytecode for the .pyc file. That is one of the parts of CPython that I simply haven't gotten to yet, but I have a hunch you know what you are talking about. :) Forgive me if I am wildly off, but in that case it would require a builder to be in a C module, or for it to tie into the existing mechanism CPython uses to build .pyc files and to use them. The former is what the implicit function builder would do.</div>
<div><br></div><div>And that is not the only complexity it would add. This would be just one more means of meta-programming that people would have to wrap their heads around (if they wanted to use it). It would also add more complexity to the C-API.</div>
<div><br></div><div>In light of all this, the benefits would have to be substantial, which is not clear to me that they are, which in reality means that it isn't worth getting into Python for now. Sometimes features have a way of coming in later when the benefits make it worth it, but I am not going to hold my breath on this one. However, the seeming flexibility of the idea is alluring. I suppose that's why it keeps coming up.</div>
<div><br></div><font color="#888888"><div>-eric</div></font><div><div></div><div class="h5"><div> </div><div><br><div class="gmail_quote">On Tue, Mar 29, 2011 at 11:40 AM, Guido van Rossum <span dir="ltr"><<a href="mailto:guido@python.org" target="_blank">guido@python.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Tue, Mar 29, 2011 at 10:26 AM, Eric Snow <<a href="mailto:ericsnowcurrently@gmail.com" target="_blank">ericsnowcurrently@gmail.com</a>> wrote:<br>
> During the discussion about assignment decorators Nick brought up the idea<br>
> of adding a def-from syntax:<br>
<br>
> def <NAME> from <BUILDER>(<ARGS>):<br>
> ...<br>
<br>
> which would effectively call<br>
<br>
> <BUILDER>(<NAME>, <ARGS>, <BODY>)<br>
<br>
> This is much like how meta-classes work, though this would be with functions<br>
> (sort of meta-functions). Ultimately, they would amount to the same thing.<br>
> The current "def" statement would just have an implicit builder.<br>
<br>
> During the sprints I explored the idea of this with Nick by making a<br>
> "build_from" decorator, and an exec_closure builtin to provide the full<br>
> capability needed to emulate the def-from syntax with a code object. It<br>
> turned out the exec_closure didn't buy much. However, one thing that became<br>
> apparent in discussing this with Nick is that just passing the code object<br>
> of the decorated function (or of the def body) as <BODY> doesn't buy much.<br>
> To really get much bang out of this you would need to pass the AST of the<br>
> body. With the AST you could manipulate it as needed before compiling.<br>
> (Nick's idea) Suddenly class definitions are just a special case of<br>
> def-from.<br>
<br>
</div>I'm glad you went down this particular rabbit hole in so much detail.<br>
You have proved beyond a doubt that the idea is not compatible with<br>
how Python currently compiles code, since it would mean that you<br>
couldn't save the generated bytecode to a .pyc file just by parsing<br>
and compiling the source code -- either you'd have to have the runtime<br>
environment available to generate the bytecode, or you'd have to put<br>
off generating the bytecode until much later.<br>
<br>
(I knew this all along, but had a hard time explaining it to the<br>
proponents of things like this, or the "make" statement, etc. -- many<br>
people have quite a misguided idea about how dynamic Python really is,<br>
and this doesn't stop them from proposing changes that only make sense<br>
in the alternate reality they believe they live in.)<br>
<div><br>
> If you really wanted to get crazy you could pass the raw string as <BODY><br>
> (can't blame this one on Nick). With the raw string you could put just<br>
> about anything in there, like a DSL or another programming language. Then<br>
> parse it however you like, and use that result to compile something else or<br>
> build some data set or call some external library or whatever you care to do<br>
> with raw data. It would not be restricted to parsable Python In the normal<br>
> "def" context Python is hardwired to turn it into a function code object,<br>
> and the implicit builder to generate the function object thereon.<br>
<br>
</div>Yeah, this is a nice reduction to the absurd of the original idea; the<br>
contradiction you've arrived at proves that the original idea cannot<br>
work.<br>
<div><br>
> I am reticent to suggest adding undue complexity to the languge or adversely<br>
> affect readability (potentially) if it doesn't offer plenty of increased<br>
> expressive power. So, crazy raw string thing aside, how about the def-from<br>
> syntax, particularly with the AST passed? Nick already indicated to me that<br>
> we probably should get comfy with metaclass __prepare__ before we get any<br>
> more metaprogramming, and he's probably right. I was thinking about<br>
> implementing def-from as an exercise in syntax hacking, regardless. Any<br>
> thoughts? Are there better syntax hacking exercises (like Raymond's "def<br>
> x.y(..." or "x.y = ..." or "x.(name)"? Could def-from have a place in the<br>
> future?<br>
<br>
</div>To me, the idea has always been dead. I'm glad you've provided the<br>
supporting documentation of its demise.<br>
<font color="#888888"><br>
--<br>
--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)<br>
</font></blockquote></div><br></div>
</div></div></blockquote></div><br></div>