On Apr 3, 2014 2:55 AM, "Andrew Barnert" <abarnert@yahoo.com> wrote:
> It's definitely a big deal. In Python 3.4, it is trivial to write a wrapper that perfectly forwards its arguments. Or that modifies its arguments in somewhat, then perfectly forwards the modified version. This is critical to being able to write decorators. Nearly every existing decorator in the stdlib, third-party libs, ActiveState recipes, and even your own projects does this. I'll show a dead-simple example below.
>
>
> But with either of your options, that would no longer be true. It would be at least a little more difficult to write a wrapper that perfectly forwards its arguments, and every existing decorator in the world would have to be rewritten to do so.

I'll concede "a little more difficult" in the pass-through case (with unaware decorators being the main issue).  :)  One reason why I don't think it's a huge problem in the decorator case is that you decorate your own functions (though not necessarily with your own decorators).  So if you change an existing function to use ordered kwargs, then you would already be focused on the feature.  You would ensure any decorators you've applied to the function accommodate ordered pass-through, perhaps adapting/wrapping the decorators to do so.  It's all pretty localized in your code, so it should be hard to miss.

I agree that both of the approaches I've mentioned require this extra work.  I also agree that it would be good to avoid this complication.  I have a few more ideas which I'll include in the upcoming PEP (and I've summarized below).

Other than the pass-through case, the 2 outstanding approaches seem fine.  Existing functions would probably need a signature change anyway and for new functions no one has to change any code. :)  Either way, if a function depends on ordered kwargs, then that's the sort of important feature that you will make clear to people using your function, on par with the rest of the signature.

[snipped example]

> Can Python fix this for you magically?

I'll drop this proposal in a heartbeat if it requires something like that!  I just don't think it does, even with the 2 less-optimal solutions we've been discussing.  (My definition of magic may be different than yours though. <wink>)

>
> The only way to make eggs end up as order-preserving is to change Python so that all functions (or at least all functions defined inside other functions) are order-preserving.
>
> Either you or someone else suggested that if Python could just pass the kwargs dict straight through instead of unpacking and repacking it, that would help. But it wouldn't.

Agreed.  (Pretty sure that wasn't me. :) )  That would be a non-option for other reasons anyway.

[more snipped]
> Or consider your original example (with explicit a and b params on spam before **kwargs) with memoize, where the kwargs in eggs has a, b, c, and d, but the one inside spam has only c and d.

That should be a non-issue given how the data is packed into a new kwargs.

>
> Now, obviously you could fix any particular wrapper. Presumably your magic decorator works by doing something detectable from Python, like a new co_flags flag. So, we could define two new functions (presumably in inspect and functools, respectively):
>
>
>     def preserves_kwargs_order(func):
>         return func.__code__.co_flags & 16
>
>     def wrap_preserve_kwargs_order(func):
>         def wrapper(inner):
>             if inspect.preserves_kwargs_order(func):
>                 return preserve_kwargs_order(inner)
>             else:
>                 return inner
>         return wrapper
>
> And now, all you have to do is add one line to most decorators and they're once again perfect forwarders:
>
>     def memoize(func):
>
>         _cache = {}
>         @functools.wrap_preserve_kwargs_order(func)
>         def wrapper(*args, **kwargs):
>             if (args, kwargs) not in _cache:
>                 _cache[args, kwargs] = func(*args, **kwargs)
>             return _cache[args, kwargs]
>         return wrapper
>
>
> But the point is, you still have to add that decorator to every wrapper function in the world or they're no longer perfect forwarders.

Nice.

>
> You could even add that wrap_preserve_kwargs_order call into functools.wraps, and that would fix _many_ decorators (because many decorators are written to use functools.wraps), but still not all.

Also nice.

That said, I agree that it would be best to come up with a solution that drops in without breaking the "perfect forwarders", thus rendering those tools unnecessary.  Here are 3 ideas I had last night and today:

1. Add a '__order__' attribute to dict that defaults to None, and always set it for kwargs (to a frozen list/tuple).
2. Use a minimal dict subclass just for kwargs that provides __order__ (meaning all other uses of dict don't take the memory hit).
3. Use OrderedDict for kwargs by default, and provide a decorator that people can use to get just a dict (a.k.a. the current behavior).  It's only a few extreme cases where OrderedDict is problematic.

-eric