[Python-ideas] Delay evaluation of annotations

[Stephen J. Turnbull]

אלעזר writes:

 > But _some_ people (medium-level, Steven, whose main language is
 > probably not Python) will not even know [function names are looked
 > up at runtime, and so the called function may not be why you think
 > it is] is the case.

And the dinosaurs will have returned by independent evolution by the
time it matters to them (unless it's a deliberate attack, in which
case people at that level would be toast anyway).

But I think you're completely missing what people are trying to tell
you.  You shouldn't be so concerned with refuting their arguments
because it doesn't matter.  No matter how many points you amass for
technique, you're going to get creamed on style points anyway.  It's
like this:

(1) Python is a "consenting adults" language, and that is presumed by
    its development culture.  The goal is not to stop people from
    creating "functions that look like recursions but aren't" on
    purpose; it's to make it easy for them to write recursive
    functions if they want to.  From your example, that goal is
    obviously satisfied.  Nobody who matters wants to go farther than
    that in Python.

    The reason one can create "functions that look like recursions but
    aren't" is because another of Python's goals is to ensure that all
    things -- specifically including functions -- are objects that can
    be manipulated "the same way" where appropriate -- in this case,
    saving off the original function object somewhere then rebinding
    the original name to something else.[1]  Granted, we don't go so
    far as Lisp where expressions are lists that you can manipulate
    like any other list, but aside from the fact that the code itself
    is an opaque object, functions are no different from other
    objects.  Even builtins:

    Python 3.6.0a4 (default, Sep  3 2016, 19:21:32) 
    >>> def help(*ignored, **paid_no_attention):
    ...  print("Ouch, you just shot off your foot!")
    ... 
    >>> help(help)
    Ouch, you just shot off your foot!
    >>> 

    Shooting off your opposite extremity by redefining builtin classes
    is left as an exercise for the reader.

    All of this is a matter of the general attitude of pragmatism and
    bias toward simplicity of implementation (both enshrined in the
    Zen of Python).

(2) You keep talking about others being lost in terminology, but in
    the context of Python discussions, you have a really big problem
    yourself.  You use the phrase "just an annotation" as though that
    means something, but there is nothing like a "just an <anything>"
    in Python discourse, not in the sense that "once we introduce
    <anythings>s, they can be anything we want".  The Language
    Reference defines what things are possible, and truly new ones are
    rarely added.

    This is deliberate.  Another design principle is Occam's Razor,
    here applied as "new kinds of thing shall not spring up like
    whiskers on Barry's chin."  Yes, function annotations need new
    syntax and so are a new kind of thing to that extent.  *Their
    values don't need to be,* and even the annotations themselves are
    implemented in the preferred way for "new things" (a dunder on an
    existing type).  Since it's new syntax, it's language-level, and
    so the values are going to be something already defined in the
    language reference.  "Expression resolving to object to be saved
    in an attribute on the function" seems to be as close to "anything
    you want" as you're gonna get without a new kind of thing.

(3) Python has a very simple model of expressions.  The compiler turns
    them into code.  The interpreter executes that code, except in the
    case where it is "quoted" by the "def" or "lambda" keywords, in
    which case it's stored in an object (and in the case of "def",
    registered in a namespace).

    As Nick admits, you could indeed argue that initializations and
    annotation values *could* consistently be turned into "thunks"
    (stored code objects, we already have those) in attributes on the
    function object.  But

    (1) that's an extension of the model (admittedly slight since
        functions, which already do that for their bodies, are
        involved -- but see Nick's reply for the hidden difficulties
        due to normal handling of namespaces in Python), and

    (2) it's a clear pessimization in the many cases where those
        values are immutable or very rarely mutated, and the use case
        (occasional) of keeping state in mutable values.  The thunk
        approach is more complex, for rather small benefit.  Re "small
        benefit", IMHO YMMV, but at least with initialization Guido is
        on record saying it's the RightThang[tm] (despite a propensity
        of new users to write buggy initializations).

(4) Chris argues that "compile to thunk" is incoherent, that
    expressions in function bodies are no different than anywhere else
    -- they're evaluated when flow of control reaches them.  AFAICS
    that *still* doesn't rule out having the compiler recognize the
    syntax and produce code that returns thunks instead of ordinary
    values, but Chris's point makes that seem way too magical to me.

(5) This points up the fact that Python is thoroughly dynamic.  It's
    not just that types adhere to objects rather than variables, but
    the whole attitude toward language design and implementation is.
    A variable not defined because it's on the path not taken, or even
    a function: they just don't exist as far as the interpreter is
    concerned -- there's no way to find them from Python.  That's not
    true in say C: if you have a powerful enough debugger, you can
    even call a function defined, but never referenced, in the code.

    So while we'd be happy for people familiar with "statically-typed
    languages" to enjoy the benefits of using Python for some of their
    work, we can't help them if they can't shake off that attitude
    when using Python.  Making things seem intuitive (which here
    translates to "familiar", as usual) to them is just misleading.
    Python doesn't work that way, and often enough, that matters.

(6) As you point out: of course, thunks are more general than values
    (in fact, without instructions that move them around, in computers
    a value is just a tree falling in a forest with noone to hear).
    But maximum generality is not necessarily an important goal, even
    if it makes some things prettier.  Allow me to quote the late
    Saunders Mac Lane: "[G]ood general theory does not search for the
    maximum generality, but for the right generality."

(7) Re Nick's comment about backward compatibility on the high bar
    having a G degree of difficulty, I'm sure you don't disagree with
    the principle of avoiding compatibility breaks.

    But while that's probably the argument that defeats this proposal
    here, I think that even looking forward from the time before
    releasing Python 3.0, the decision would be the same.  That is, I
    think the decision to go with the simpler "evaluate to object"
    model was one of the right decisions at that time, for the reasons
    above.  Your proposal of "evaluate to thunk" (possibly
    incorporating the property-based magic Alexander proposed) might
    be right *too*, but it's far from obviously better to me.  I see
    nothing there that would be likely to have dissuaded the authors
    of PEP 3107, or Guido when he designed default initialization,
    from "evaluate to object".

If you don't like that philosophy, or somehow don't think it applies
here, keep trying, you may have a point.  But in this thread, IMO
you're trying to ski up a slope that's barely able to hold snow, and
wasting everybody's time.  And even if I'm wrong about wasting time
with the feature, you'll be way more persuasive if you argue in terms
of Python as it is designed (mostly deliberately so), which is the way
most of us mostly like it.  Although we do change our mind every 18
months. :-)


Footnotes: 
[1]  For maximum humor, rebind it to a different recursive function!