[Python-3000] PEP 3124 - more commentary

[Guido van Rossum]

On 5/14/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:25 AM 5/14/2007 -0700, Guido van Rossum wrote:
> >The implementation of @overload needs to use sys._getframe() to look
> >up the name of the function ('flatten') in the surrounding namespace.
> >I find this too fragile an approach; it means that I can't easily
> >write another function that calls overload to get the same effect; in
> >particular, I don't see how this code could work:
> >
> >   def my_overload(func):
> >     "Shorthand for @some_decorator + @overload."
> >     return some_decorator(overload(func))
> >
> >   @my_oveload
> >   def flatten(z: int): ...
> >
> >If the overload decorator simply looked in the calling scope, it would
> >not find 'flatten' there, since that's the local scope of my_overload.
> >(If it devised some clever scheme of descending down the stack, I
> >would just have to create a more complicated example.)
>
> Actually, your "my_overload" would just need to do its own getframe
> and call when() on the result, since overload is just sugar for when().

That does nothing to address my abhorrence of sys._getframe(). To the
contary, it looks like knowledge of the implementation is required for
proper use and understanding of @overload. A big fat -1 on that.

> >I realize that @overload is only a shorthand for @when(function). But
> >I'd much rather not have @overload at all -- the frame inspection
> >makes it really hard for me to explain carefully what happens without
> >just giving the code that uses sys._getframe(); and this makes it
> >difficult to reason about code using @overload.
>
> This is why in the very earliest GF discussions here, I proposed a
> 'defop expr(...)' syntax, as it would eliminate the need for any
> getframe hackery.

But that would completely kill your "but it's all pure Python code so
it's harmless and portable" argument.

It seems that you're really not interested at all in compromising to
accept mandatory marking of the base overloadable function. That's too
bad, because I'm not compromising either *on that particular issue*.

> >My own preference for spelling this example would be
> >
> >@overloadable
> >def flatten(x): ...
> >
> >@flatten.overload
> >def _(y: str): ...
>
> Btw, this is similar to how RuleDispatch actually spells it, except
> that it's @flatten.when().  Later, I decided I preferred putting the
> *mode* of combination (e.g. when vs. around vs. whatever) first, both
> because it reads more naturally (e.g. "when flattening", "before
> flattening", etc.)

But the function name isn't "flattening" (and there are good reasons
for that). This requires too much squinting to work.

> and because it enabled one to retroactively extend
> existing functions.

Which as you know I don't like, so that argument doesn't hold.

I find that "when" feels like a "condition" (albeit a temporal one)
and I'd much rather read the descriptor in terms of what the action of
the decorator is (i.e. some kind of registration) rather than trying
to read like some vaguely declarative English phrase.

> >Next, I have a question about the __proceed__ magic argument. I can
> >see why this is useful, and I can see why having this as a magic
> >argument is preferable over other solutions (I couldn't come up with a
> >better solution, and believe me I tried :-).  However, I think making
> >this the *first* argument would upset tools that haven't been taught
> >about this yet. Is there any problem with making it a keyword argument
> >with a default of None, by convention to be placed last?
>
> Actually, a pending revision to the PEP is to drop the special name
> and instead use a special annotation, e.g.:
>
>      def whatever(nm:next_method, ...):
>
> (This idea came up in an early thread when some folks queried whether
> a better name than __proceed__ could be found.)

Cool, I agree that an annotation is better than a magic name.

> Anyway, with this, it could also be placed as a keyword
> argument.  The main reason for putting it in the first position is
> performance.  Allowing it to be anywhere, however, would let the
> choice of where be a matter of style.

Right. What's the performance issue with the first argument?

> >Finally, I looked at the example of overloading a method instead of a
> >function.  The little dance required to overload a method defined in a
> >base class feels fragile,
>
> Note that a defop syntax would simplify this; i.e. :
>
>      defop MyBaseClass.methodname(...):
>          ...
>
> This doesn't help with the first-argument magic, however.
>
> However, since we're going to have to have some way for 'super' to
> know the class a function is defined in, ISTM that the same magic
> should be reusable for the first-argument rule.

Perhaps. Though super only needs to know it once the method is being
called, while your decorator (presumably) needs to know when the
method is being defined, i.e. before the class object is constructed.

Also, the similarities between next-method and super are overwhelming.
It would be great if you could work with Tim Delaney on a mechanism
underlying all three issues, or at least two of the three.

> >Forgive me if this is mentioned in the PEP, but what happens with
> >keyword args? Can I invoke an overloaded function with (some) keyword
> >args, assuming they match the argument names given in the default
> >implementation?
>
> Yes.  That's done with code generation; PEAK-Rules uses direct
> bytecode generation, but a sourcecode-based generation is also
> possible and would be used for the PEP implementation (it was also
> used in RuleDispatch).

There's currently no discussion of this. Without a good understanding
of the implementation I cannot accept the PEP.

> >Also, can we overload different-length signatures (like in C++ or
> >Java)? This is very common in those languages; while Python typically
> >uses default argument values, there are use cases that don't easily
> >fit in that pattern (e.g. the signature of range()).
>
> I see a couple different possibilities for this.  Could you give an
> example of how you'd *like* it to work?

In the simplest case (no default argument values) overloading two-arg
functions and three-arg functions with the same name should act as if
there were two completely separate functions, except for the base
(default) function. Example:

@overloadable
def range(start:int, stop:int, step:int):
  ...  # implement xrange

@range.overload
def range(x): return range(0, x, 1)

@range.overload
def range(x, y): return range(x, y, 1)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)