[Python-ideas] def-from

Tue Mar 29 20:58:54 CEST 2011

I appreciate your candor.  You are right that the bytecode would have to
live elsewhere, probably memory.  Or you would have to handle the .pyc
generation dynamically.  namedtuples do dynamic code generation and
execution, but not on the same scale as def-from would cause.

Unless I misunderstand (not unlikely), it seems like your objection is that
the code for the builder in a def-from would not necessarily be around yet
to build the bytecode for the .pyc file.  That is one of the parts of
CPython that I simply haven't gotten to yet, but I have a hunch you know
what you are talking about. :)   Forgive me if I am wildly off, but in that
case it would require a builder to be in a C module, or for it to tie into
the existing mechanism CPython uses to build .pyc files and to use them.
 The former is what the implicit function builder would do.

And that is not the only complexity it would add.  This would be just one
more means of meta-programming that people would have to wrap their heads
around (if they wanted to use it).  It would also add more complexity to the
C-API.

In light of all this, the benefits would have to be substantial, which is
not clear to me that they are, which in reality means that it isn't worth
getting into Python for now.  Sometimes features have a way of coming in
later when the benefits make it worth it, but I am not going to hold my
breath on this one.  However, the seeming flexibility of the idea is
alluring.  I suppose that's why it keeps coming up.

-eric

On Tue, Mar 29, 2011 at 11:40 AM, Guido van Rossum <guido at python.org> wrote:

> On Tue, Mar 29, 2011 at 10:26 AM, Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
> > During the discussion about assignment decorators Nick brought up the
> idea
> > of adding a def-from syntax:
>
> > def <NAME> from <BUILDER>(<ARGS>):
> >     ...
>
> > which would effectively call
>
> > <BUILDER>(<NAME>, <ARGS>, <BODY>)
>
> > This is much like how meta-classes work, though this would be with
> functions
> > (sort of meta-functions).  Ultimately, they would amount to the same
> thing.
> >  The current "def" statement would just have an implicit builder.
>
> > During the sprints I explored the idea of this with Nick by making a
> > "build_from" decorator, and an exec_closure builtin to provide the full
> > capability needed to emulate the def-from syntax with a code object.  It
> > turned out the exec_closure didn't buy much.  However, one thing that
> became
> > apparent in discussing this with Nick is that just passing the code
> object
> > of the decorated function (or of the def body) as <BODY> doesn't buy
> much.
> >  To really get much bang out of this you would need to pass the AST of
> the
> > body.  With the AST you could manipulate it as needed before compiling.
> >  (Nick's idea)  Suddenly class definitions are just a special case of
> > def-from.
>
> I'm glad you went down this particular rabbit hole in so much detail.
> You have proved beyond a doubt that the idea is not compatible with
> how Python currently compiles code, since it would mean that you
> couldn't save the generated bytecode to a .pyc file just by parsing
> and compiling the source code -- either you'd have to have the runtime
> environment available to generate the bytecode, or you'd have to put
> off generating the bytecode until much later.
>
> (I knew this all along, but had a hard time explaining it to the
> proponents of things like this, or the "make" statement, etc. -- many
> people have quite a misguided idea about how dynamic Python really is,
> and this doesn't stop them from proposing changes that only make sense
> in the alternate reality they believe they live in.)
>
> > If you really wanted to get crazy you could pass the raw string as <BODY>
> > (can't blame this one on Nick).  With the raw string you could put just
> > about anything in there, like a DSL or another programming language.
>  Then
> > parse it however you like, and use that result to compile something else
> or
> > build some data set or call some external library or whatever you care to
> do
> > with raw data.  It would not be restricted to parsable Python  In the
> normal
> > "def" context Python is hardwired to turn it into a function code object,
> > and the implicit builder to generate the function object thereon.
>
> Yeah, this is a nice reduction to the absurd of the original idea; the
> contradiction you've arrived at proves that the original idea cannot
> work.
>
> > I am reticent to suggest adding undue complexity to the languge or
> adversely
> > affect readability (potentially) if it doesn't offer plenty of increased
> > expressive power.  So, crazy raw string thing aside, how about the
> def-from
> > syntax, particularly with the AST passed?  Nick already indicated to me
> that
> > we probably should get comfy with metaclass __prepare__ before we get any
> > more metaprogramming, and he's probably right.  I was thinking about
> > implementing def-from as an exercise in syntax hacking, regardless.  Any
> > thoughts?  Are there better syntax hacking exercises (like Raymond's "def
> > x.y(..." or "x.y = ..." or "x.(name)"?  Could def-from have a place in
> the
> > future?
>
> To me, the idea has always been dead. I'm glad you've provided the
> supporting documentation of its demise.
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20110329/8a142e0a/attachment.html>