[Python-ideas] Syntax for easy binding __name__, __module__, __qualname__ to arbitrary objects

[Nick Coghlan]

On Tue, May 14, 2013 at 10:03 AM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Martin Morrison wrote:
>>
>> Def is not a constructor. It is an assignment statement.
>
>
> It's really some of both. It constructs a function
> object, and binds it to a name. These two operations
> are entwined, in a sense, because the name being
> bound to is used as an argument in the construction.
>
> We're after a generalisation of this entwined
> construction-and-assignment. (Astruction? Consignment?)
> I'm not entirely happy with the current proposal:
>
>    def name = expr
>
> because it doesn't fully entwine. The expr can be a
> constructor, but doesn't have to be, and even when it
> is, the construction occurs separately from the
> assignment. Also, it looks like an ordinary assignment
> with 'def' stuck in front, which, as Guido points
> out, seems somewhat random.

It doesn't seem random to me, and it ties into some conceptual issues
I've been pondering for a long time. I see this as potentially related
to PEP 395 and the way __main__ and pseudo-modules are prone to
breaking pickle and exposing implementation details, as well as to the
fact we added @functools.wraps to preserve the original location
details of decorated functions. It also ties into Guido's rationale
for wanting to preserve the way that namedtuple (and now enum) special
case simple module level assignments to help ensure that pickling
works as expected (see http://bugs.python.org/issue17947#msg189160)

What Piotr's proposal crystalised for me is the idea that we really
have two different kinds of name binding in Python. I'm going to call
them "incidental binding" and "definitive binding".

Incidental binding is the way we normally think about variables in
Python - it's just a local name that is used to reference an object.
Objects know nothing about their incidental bindings, and (aside from
function parameters and keyword arguments) the specific name used is
typically of no interest outside the scope where the binding happens.

For loops, with statements, except clauses, assignment statements -
these all create incidental bindings, where the target name and the
location of the binding is of no interest to the object being bound.

By contrast, definitive bindings are exactly those where the object
being bound *cares* about the name and where it happens. The most
obvious case where the definitive binding matters is pickle
compatibility, because the definitive name is what gets used to
retrieve the appropriate code when unpickling. However, it also
affects introspection, as the existence of @functools.wraps shows.

def statements and class statements are definitive bindings - through
the constructor, they set __name__, __qualname__ and __module__ on the
bound object based on the location of the statement itself.
@functools.wraps works because it converts a normally definitive
binding (the declaration of a wrapper function) into an incidental
binding by copying the definitive binding details from the function
being wrapped.

What the frame introspection hack in namedtuple and enum achieves is
to allow an ordinary module level assignment to be treated as
definitive - these APIs require that the name be passed in explicitly,
but if the module is omitted they can look it up in the globals of the
calling frame (Guido proposes that we provider a nicer API for getting
that dynamic context information, and I'm now persuaded that he's
right, but that's independent of the underlying conceptual issue).

The reason I'm a big fan of Piotr's idea in general, and the "def NAME
= EXPR" syntax in particular is that I think it takes this currently
implicit, nebulous concept and elevates it to the status it deserves:
giving an object an incidental label for convenient local reference
and defining the *canonical* name for that object are different
operations, and it is reasonable to provide an explicit syntax for the
latter that is distinct-from-but-related-to the syntax for the former.

Specifically, as Piotr suggested, I would like to see "def <NAME> =
<EXPR>" become syntactic sugar for something like:

    # We start with an ordinary incidental binding
    <NAME> = _ref = <EXPR>
    if hasattr(_ref, "__defname__"):
        # We elevate this to a definitive binding by passing
        # __name__, __qualname__ and __module__ to the bound object
        _ref.__defname__("<NAME>", <QUALIFER> + "<NAME>", __name__)

The beauty of this syntax is that it means if we define __defname__
appropriately on function objects and on type, then we can cleanly
separate the "object creation" step from the "name binding" step,
allowing functional APIs to exist on an equal playing field with the
dedicated syntax.

Such a change would also help explain why we *don't* allow arbitrary
assignment targets in a def statement - as a definitive name binding,
there are additional constraints that don't apply to an incidental
binding.

> This reads quite naturally: "define Animal as an
> Enum with these arguments."

Whereas I prefer the idea that the RHS is created as an anonymous
object, and then bound to a canonical name.

> Another example based on my own use case:
>
>    def width as overridable_property("The width of the widget.")
>
> (Yes, this is yet another, different use of the word "as",
> but I don't see anything wrong with that. Small words in
> English often don't mean much on their own and derive most
> of their meaning from their context.)

Strong -1. We've had this discussion before, and any use of "as"
should be to bind to a name on the RHS and the value bound should
*not* be the expression on the LHS, but some related object (an
imported module, a caught exception, the result of an __enter__
method). If the expression is being bound directly, then the name
should appear on the LHS and use "=" as the symbol.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia