Re: [Python-ideas] Preserving **kwargs order
On Apr 2, 2014, at 23:39, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Thu, Mar 20, 2014 at 10:28 AM, Andrew Barnert <abarnert@yahoo.com> wrote:
And as far as I can see, there is no way to solve this problem. The only way to add keyword order information without breaking all existing forwarding functions is to make the **kwargs parameter always preserve order (and make the **kwargs calling syntax preserve order if present)--that is, to make it always an OrderedDict, which Guido has already rejected.
It's not all that bleak. Either of the options I outlined should still work, though option #1 (using a decorator) is better for the pass-through case that has come up (which I'm not convinced is such a big deal). It's definitely a big deal. In Python 3.4, it is trivial to write a wrapper that perfectly forwards its arguments. Or that modifies its arguments in somewhat, then perfectly forwards the modified version. This is critical to being able to write decorators. Nearly every existing decorator in the stdlib, third-party libs, ActiveState recipes, and even your own projects does this. I'll show a dead-simple example below.
But with either of your options, that would no longer be true. It would be at least a little more difficult to write a wrapper that perfectly forwards its arguments, and every existing decorator in the world would have to be rewritten to do so. Let's take your specific example, and simplify it even further. I'll show why it doesn't _obviously_ work, and then you try to explain how you could make it work anyway: @preserve_kwargs_order def spam(**kwargs): print(kwargs) spam(a=1, b=2) So far, so good; your proposal means spam is guaranteed to get an OrderedDict with a before b. Now let's take a simple and useful general-purpose wrapper: def memoize(func): cache = {} def wrapper(*args, **kwargs): if (args, kwargs) not in _cache: cache[args, kwargs] = func(*args, **kwargs) return cache[args, kwargs] return wrapper eggs = memoize(spam) eggs(a=1, b=2) Here, spam is guaranteed to get an OrderedDict, but it's arbitrary whether a comes before or after b. Why? Well, inside eggs (aka wrapper), kwargs is a plain old dict. So when it calls spam (aka func) with **kwargs, they get sent in whatever arbitrary order they had in the dict. Would anyone ever write or use a wrapper like this? Well, the Python developers thought it was useful enough to add a more complicated version of the same thing to the stdlib as functools.lru_cache, and there were dozens of recipes and third-party modules doing the same thing before it was added to the stdlib, so, yes. And, more importantly, if you work through any other general-purpose wrapper, it's going to have the exact same problems. (It's marginally harder to work through, but maybe more obvious once you do so, with wrappers that modify their arguments, like functools.partial.) Can Python fix this for you magically? There are four potential places it could, and none of them will work. When compiling memoize, Python obviously has no idea whether the value (or values) you will later pass to it are order-preserving functions or normal functions. At runtime, when executing that compiled memoize definition, you still have the same problems. Later, when calling memoize on spam, Python now knows that the value is an order-preserving function—but it no longer knows that memoize is intended to be a perfect-forwarding wrapper-generator. (In fact, there is nothing special about the definition of memoize in the first place that makes it detectably a perfect-forwarding wrapper at any point. If that isn't obvious for memoize, consider functools.lru_cache or functools.partial.) And finally, when calling the memoized function eggs, it's obviously still too late.So, there is no place that Python could automatically figure out that it needs to push the order-preservation upward from spam to eggs. The only way to make eggs end up as order-preserving is to change Python so that all functions (or at least all functions defined inside other functions) are order-preserving. Either you or someone else suggested that if Python could just pass the kwargs dict straight through instead of unpacking and repacking it, that would help. But it wouldn't. There's nothing marking eggs as an order-preserving function, so its kwargs is just a plain dict, and passing that straight through can't help anything. And besides, that idea isn't possible, or even (in general) sensible. Consider eggs=partial(spam, c=3), where the kwargs in eggs only has a and b, but the one inside spam has a, b, and c; obviously they can't be the same mapping. Or consider your original example (with explicit a and b params on spam before **kwargs) with memoize, where the kwargs in eggs has a, b, c, and d, but the one inside spam has only c and d. Now, obviously you could fix any particular wrapper. Presumably your magic decorator works by doing something detectable from Python, like a new co_flags flag. So, we could define two new functions (presumably in inspect and functools, respectively): def preserves_kwargs_order(func): return func.__code__.co_flags & 16 def wrap_preserve_kwargs_order(func): def wrapper(inner): if inspect.preserves_kwargs_order(func): return preserve_kwargs_order(inner) else: return inner return wrapper And now, all you have to do is add one line to most decorators and they're once again perfect forwarders: def memoize(func): _cache = {} @functools.wrap_preserve_kwargs_order(func) def wrapper(*args, **kwargs): if (args, kwargs) not in _cache: _cache[args, kwargs] = func(*args, **kwargs) return _cache[args, kwargs] return wrapper But the point is, you still have to add that decorator to every wrapper function in the world or they're no longer perfect forwarders. You could even add that wrap_preserve_kwargs_order call into functools.wraps, and that would fix _many_ decorators (because many decorators are written to use functools.wraps), but still not all. Most notably, the wrappers inside functools don't use wraps (even if you don't get the C-accelerated versions). So, that would still means that you have to add the new decorator to every wrapper function in the world that doesn't use wraps.
On Apr 3, 2014 2:55 AM, "Andrew Barnert" <abarnert@yahoo.com> wrote:
It's definitely a big deal. In Python 3.4, it is trivial to write a wrapper that perfectly forwards its arguments. Or that modifies its arguments in somewhat, then perfectly forwards the modified version. This is critical to being able to write decorators. Nearly every existing decorator in the stdlib, third-party libs, ActiveState recipes, and even your own projects does this. I'll show a dead-simple example below.
But with either of your options, that would no longer be true. It would be at least a little more difficult to write a wrapper that perfectly forwards its arguments, and every existing decorator in the world would have to be rewritten to do so.
I'll concede "a little more difficult" in the pass-through case (with unaware decorators being the main issue). :) One reason why I don't think it's a huge problem in the decorator case is that you decorate your own functions (though not necessarily with your own decorators). So if you change an existing function to use ordered kwargs, then you would already be focused on the feature. You would ensure any decorators you've applied to the function accommodate ordered pass-through, perhaps adapting/wrapping the decorators to do so. It's all pretty localized in your code, so it should be hard to miss. I agree that both of the approaches I've mentioned require this extra work. I also agree that it would be good to avoid this complication. I have a few more ideas which I'll include in the upcoming PEP (and I've summarized below). Other than the pass-through case, the 2 outstanding approaches seem fine. Existing functions would probably need a signature change anyway and for new functions no one has to change any code. :) Either way, if a function depends on ordered kwargs, then that's the sort of important feature that you will make clear to people using your function, on par with the rest of the signature. [snipped example]
Can Python fix this for you magically?
I'll drop this proposal in a heartbeat if it requires something like that! I just don't think it does, even with the 2 less-optimal solutions we've been discussing. (My definition of magic may be different than yours though. <wink>)
The only way to make eggs end up as order-preserving is to change Python
so that all functions (or at least all functions defined inside other functions) are order-preserving.
Either you or someone else suggested that if Python could just pass the
kwargs dict straight through instead of unpacking and repacking it, that would help. But it wouldn't. Agreed. (Pretty sure that wasn't me. :) ) That would be a non-option for other reasons anyway. [more snipped]
Or consider your original example (with explicit a and b params on spam before **kwargs) with memoize, where the kwargs in eggs has a, b, c, and d, but the one inside spam has only c and d.
That should be a non-issue given how the data is packed into a new kwargs.
Now, obviously you could fix any particular wrapper. Presumably your
magic decorator works by doing something detectable from Python, like a new co_flags flag. So, we could define two new functions (presumably in inspect and functools, respectively):
def preserves_kwargs_order(func): return func.__code__.co_flags & 16
def wrap_preserve_kwargs_order(func): def wrapper(inner): if inspect.preserves_kwargs_order(func): return preserve_kwargs_order(inner) else: return inner return wrapper
And now, all you have to do is add one line to most decorators and
they're once again perfect forwarders:
def memoize(func):
_cache = {} @functools.wrap_preserve_kwargs_order(func) def wrapper(*args, **kwargs): if (args, kwargs) not in _cache: _cache[args, kwargs] = func(*args, **kwargs) return _cache[args, kwargs] return wrapper
But the point is, you still have to add that decorator to every wrapper
function in the world or they're no longer perfect forwarders. Nice.
You could even add that wrap_preserve_kwargs_order call into
functools.wraps, and that would fix _many_ decorators (because many decorators are written to use functools.wraps), but still not all. Also nice. That said, I agree that it would be best to come up with a solution that drops in without breaking the "perfect forwarders", thus rendering those tools unnecessary. Here are 3 ideas I had last night and today: 1. Add a '__order__' attribute to dict that defaults to None, and always set it for kwargs (to a frozen list/tuple). 2. Use a minimal dict subclass just for kwargs that provides __order__ (meaning all other uses of dict don't take the memory hit). 3. Use OrderedDict for kwargs by default, and provide a decorator that people can use to get just a dict (a.k.a. the current behavior). It's only a few extreme cases where OrderedDict is problematic. -eric
From: Eric Snow <ericsnowcurrently@gmail.com> Sent: Thursday, April 3, 2014 4:37 PM
On Apr 3, 2014 2:55 AM, "Andrew Barnert" <abarnert@yahoo.com> wrote:
It's definitely a big deal. In Python 3.4, it is trivial to write a wrapper that perfectly forwards its arguments. Or that modifies its arguments in somewhat, then perfectly forwards the modified version. This is critical to being able to write decorators. Nearly every existing decorator in the stdlib, third-party libs, ActiveState recipes, and even your own projects does this. I'll show a dead-simple example below.
But with either of your options, that would no longer be true. It would be at least a little more difficult to write a wrapper that perfectly forwards its arguments, and every existing decorator in the world would have to be rewritten to do so.
I'll concede "a little more difficult" in the pass-through case (with unaware decorators being the main issue). :) One reason why I don't think it's a huge problem in the decorator case is that you decorate your own functions (though not necessarily with your own decorators). So if you change an existing function to use ordered kwargs, then you would already be focused on the feature. You would ensure any decorators you've applied to the function accommodate ordered pass-through, perhaps adapting/wrapping the decorators to do so. It's all pretty localized in your code, so it should be hard to miss.
First, not all wrappers are usually used as decorators. The most obvious example is partial, which is why I raised it before. I've also applied lru_cache, or a TTL-based cache that I got from some module on PyPI, to lookup functions that I pulled out of another module. And so on. Any such general-purpose wrappers that work today would need to be replaced after your change. It's not at all localized in my code; it's not even all in my code. The guy who wrote TTLCache has no idea whether I want to use it for keyword-order-preserving functions. (Well, today, he knows that I _don't_ want to do so, because it's not possible…)
Other than the pass-through case, the 2 outstanding approaches seem fine. Existing functions would probably need a signature change anyway and for new functions no one has to change any code. :) Either way, if a function depends on ordered kwargs, then that's the sort of important feature that you will make clear to people using your function, on par with the rest of the signature.
Well, that has nothing to do with the point I raised, but actually, it raises another (smaller) problem. The fact that a function has been wrapped in a decorator doesn't change its signature, as in the thing you can inspect with things like functools.getfullargspec or functools.signature. So, just adding this decorator, or even adding an inspect function that looks for whatever the decorator does (like setting a new bit on co_flags), isn't sufficient; you need to work out how it should affect inspect.Signature and/or inspect.Parameter objects. (Maybe there's just a new kind, VAR_ORDERED_KEYWORD… but the fact that Parameters of this kind would have to have the same str() as VAR_KEYWORD means most people still wouldn't see it…)
Can Python fix this for you magically?
I'll drop this proposal in a heartbeat if it requires something like that! I just don't think it does, even with the 2 less-optimal solutions we've been discussing. (My definition of magic may be different than yours though. <wink>)
Well, you said before that the perfect-forwarding wrapper problem could be solved easily. I asked you to explain how. Now it seems like your answer is that it can't be solved, short of by rewriting every wrapper function that might ever be used on order-preserving functions. Which is exactly what I said, and you denied, in the first place.
But the point is, you still have to add that decorator to every wrapper function in the world or they're no longer perfect forwarders.
Nice.
The decorator may be nice, but the fact that you have to add that decorator to ever wrapper function defined in every general-purpose decorator or other wrapper-generator in the world does not seem nice to me.
That said, I agree that it would be best to come up with a solution that drops in without breaking the "perfect forwarders", thus rendering those tools unnecessary. Here are 3 ideas I had last night and today:
1. Add a '__order__' attribute to dict that defaults to None, and always set it for kwargs (to a frozen list/tuple).
A dict with an __order__ attrib is just a less-useful but nearly-as-heavy-weight equivalent to an OrderedDict. What about the cost of always using an OrderedDict makes it unacceptable? How much better is this? And it doesn't actually work. First, the **d calling syntax unpacks the keywords in d's iteration order. That works fine for OrderedDict, but not for a dict that has an order but ignores it while iterating. So you need an additional rule (and additional code in each implementation) to make the **d calling syntax use __order__ order instead of iteration order (and deal with all of the edge cases, like when __order__ is missing some keys, or has keys the dict doesn't, etc.). And, even after that, any wrapper that modifies its kwargs instead of passing them through unchanged (like, again, partial) could easily break __order__ unless you also change all such wrappers to handle it properly. Again, not a problem for OrderedDict, because OrderedDict automatically handles it properly.
2. Use a minimal dict subclass just for kwargs that provides __order__ (meaning all other uses of dict don't take the memory hit).
That still has the other problems as #1.
3. Use OrderedDict for kwargs by default, and provide a decorator that people can use to get just a dict (a.k.a. the current behavior). It's only a few extreme cases where OrderedDict is problematic.
So how do C functions (which I'm guessing are many of those extreme cases…) get just a dict? Also, is it actually true that Guido rejected the idea of "kwargs is always an OrderedDict" out of hand because a few extreme cases are problematic, or is that just a guess?
On Thu, Apr 3, 2014 at 7:50 PM, Andrew Barnert <abarnert@yahoo.com> wrote:
First, not all wrappers are usually used as decorators. The most obvious example is partial, which is why I raised it before. I've also applied lru_cache, or a TTL-based cache that I got from some module on PyPI, to lookup functions that I pulled out of another module. And so on. Any such general-purpose wrappers that work today would need to be replaced after your change. It's not at all localized in my code; it's not even all in my code. The guy who wrote TTLCache has no idea whether I want to use it for keyword-order-preserving functions. (Well, today, he knows that I _don't_ want to do so, because it's not possible...)
So you are saying that it is not uncommon to use "perfect passthrough" decorators/wrappers you do not control. In my experience it is rare outside the stdlib, but then again I gladly admit that my experience has not involved software domains where apparently such takes place. Regardless, the workarounds are the same. Likewise they add the same complication. Perhaps it does favor the decorator syntax over the __kworder__ local variable.
Well, that has nothing to do with the point I raised, but actually, it raises another (smaller) problem. The fact that a function has been wrapped in a decorator doesn't change its signature, as in the thing you can inspect with things like functools.getfullargspec or functools.signature.
So, just adding this decorator, or even adding an inspect function that looks for whatever the decorator does (like setting a new bit on co_flags), isn't sufficient; you need to work out how it should affect inspect.Signature and/or inspect.Parameter objects. (Maybe there's just a new kind, VAR_ORDERED_KEYWORD... but the fact that Parameters of this kind would have to have the same str() as VAR_KEYWORD means most people still wouldn't see it...)
Looking into the implications on inspect.signature, et al. is on my to-do list. :)
Well, you said before that the perfect-forwarding wrapper problem could be solved easily. I asked you to explain how. Now it seems like your answer is that it can't be solved, short of by rewriting every wrapper function that might ever be used on order-preserving functions. Which is exactly what I said, and you denied, in the first place.
I stand by what I said. I don't think the impact is that severe and I think that solving the issue wouldn't be that hard and I do agree that it adds complication that would be worth avoiding. I think your summary exaggerates things a bit. You are correct that I did not give any concrete solution to resolve the perfect-forwarding wrapper case, even in the face of your concrete example. If, when we have a better idea of the various options, one of the original two (or any other opt-in solution) stands above the rest I'll be glad to spend more time looking on how to make the pass-through case work with less complication.
But the point is, you still have to add that decorator to every wrapper function in the world or they're no longer perfect forwarders.
Nice.
The decorator may be nice, but the fact that you have to add that decorator to ever wrapper function defined in every general-purpose decorator or other wrapper-generator in the world does not seem nice to me.
It only matters for the ones you are applying to your functions that need to preserve keyword order. I agree that any fix that you can't easily apply to a function you do not "own" isn't very helpful.
That said, I agree that it would be best to come up with a solution that drops in without breaking the "perfect forwarders", thus rendering those tools unnecessary. Here are 3 ideas I had last night and today:
1. Add a '__order__' attribute to dict that defaults to None, and always set it for kwargs (to a frozen list/tuple).
A dict with an __order__ attrib is just a less-useful but nearly-as-heavy-weight equivalent to an OrderedDict. What about the cost of always using an OrderedDict makes it unacceptable? How much better is this?
Guido's concern was that in some uncommon cases the difference in performance (both memory and speed) would render OrderedDict undesirable for **kwargs. By storing the initial order on a dict attribute the only overhead would be 1 pointer per dict plus the size of the container.
And it doesn't actually work.
First, the **d calling syntax unpacks the keywords in d's iteration order. That works fine for OrderedDict, but not for a dict that has an order but ignores it while iterating. So you need an additional rule (and additional code in each implementation) to make the **d calling syntax use __order__ order instead of iteration order (and deal with all of the edge cases, like when __order__ is missing some keys, or has keys the dict doesn't, etc.).
And, even after that, any wrapper that modifies its kwargs instead of passing them through unchanged (like, again, partial) could easily break __order__ unless you also change all such wrappers to handle it properly. Again, not a problem for OrderedDict, because OrderedDict automatically handles it properly.
Correct, the interpreter would have to take __order__ into account when unpacking kwargs into a function call. I realized earlier that I hadn't mentioned that.
2. Use a minimal dict subclass just for kwargs that provides __order__ (meaning all other uses of dict don't take the memory hit).
That still has the other problems as #1.
3. Use OrderedDict for kwargs by default, and provide a decorator that people can use to get just a dict (a.k.a. the current behavior). It's only a few extreme cases where OrderedDict is problematic.
So how do C functions (which I'm guessing are many of those extreme cases...) get just a dict?
Good question. I already plan on addressing the impact of the proposal on the C-API as part of the PEP.
Also, is it actually true that Guido rejected the idea of "kwargs is always an OrderedDict" out of hand because a few extreme cases are problematic, or is that just a guess?
He indicated that any solution would have to demonstrate that it did not have any significant impact on performance in those edge cases. [1][2][3] -eric [1] https://mail.python.org/pipermail/python-dev/2013-May/126328.html [2] https://mail.python.org/pipermail/python-ideas/2013-February/019699.html [3] https://mail.python.org/pipermail/python-ideas/2013-February/019704.html
From: Eric Snow <ericsnowcurrently@gmail.com> Sent: Thursday, April 3, 2014 8:03 PM
On Thu, Apr 3, 2014 at 7:50 PM, Andrew Barnert <abarnert@yahoo.com> wrote:
[snip]
So you are saying that it is not uncommon to use "perfect passthrough" decorators/wrappers you do not control.
Yes. Just my first example, partial, is ridiculously common, and it's a wrapper I don't control. Half the servers, GUIs, and other callback-based programs I've worked on need partial functions, and get them either by using partial, or by using lambda functions instead—which has an equivalent and parallel problem that, come to think of it, still requires a solution too… However, what we _don't_ know is how common it would be to use such wrappers with functions that need ordered kwargs. The prototypical case of using partial on GUI or server callbacks obviously doesn't apply there—in a Tkinter GUI, your callback functions almost never take kwargs, and, if they did, it would probably be just to pass the dict as a widget configure dict or something like that, which couldn't possibly care about the order. But really, _no_ example that works today could apply; it's not possible to write functions that care about their keyword order, so we can only try to guess whether the potential use cases for them could involve wrapping them or not.
Regardless, the workarounds are the same. Likewise they add the same complication. Perhaps it does favor the decorator syntax over the __kworder__ local variable.
I'm not sure it favors either. Either way, every function that returns a forwarding wrapper will need to be changed to apply the workaround. The workaround itself may be easier, or at least prettier, with the decorator, but the hard part isn't the workaround itself (as I showed, it could just be a simple on-line change), but figuring out which wrappers need it, and possibly copying and forking them into your own code, so you can apply the workaround to each one. [snip]
Looking into the implications on inspect.signature, et al. is on my to-do list. :)
After a little more thought, what's really needed is a consistent way to represent ordered-kwargs parameters in *documentation*. Once you have that, whatever it is, inspect.signature should be simple. For example, if you use kind=VAR_ORDERED_KEYWORD, then it's just a matter of making Parameter.__str__ return whatever-it-is that the docs would have instead of returning the normal **{name}. This does give a little more weight to proposals that do something visible in the signature itself—the ***okwargs, or an annotation, or whatever. But I think all of them have enough down-sides compared to your more favored versions that it wouldn't be nearly enough weight to shift things. [snip]
It only matters for the ones you are applying to your functions that need to preserve keyword order. I agree that any fix that you can't easily apply to a function you do not "own" isn't very helpful.
Yes, that last part is basically my main point.
1. Add a '__order__' attribute to dict that defaults to None, and always set it for kwargs (to a frozen list/tuple).
A dict with an __order__ attrib is just a less-useful but nearly-as-heavy-weight equivalent to an OrderedDict. What about the cost of always using an OrderedDict makes it unacceptable? How much better is this?
Guido's concern was that in some uncommon cases the difference in performance (both memory and speed) would render OrderedDict undesirable for **kwargs. By storing the initial order on a dict attribute the only overhead would be 1 pointer per dict plus the size of the container.
First, after reading the messages you linked, I don't think your characterizing Guido's concerns fairly. His first message is not about uncommon cases at all, it's about the fact that every function call (at least every one that uses kwargs) will slow down noticeably: "My main concern is speed -- since most code doesn't need it and function calls are already slow (and obviously very common :-) it would be a shame if this slowed down function calls that don't need it noticeably." It's hard to predict how much of a slowdown a nice C OrderedDict would cause here, and how much less of a slowdown just creating and using __order__ would cause. But from a quick simulation in pure Python, it looks like you're just turning a 2.7x slowdown into a 2.2x slowdown—better, but still clearly noticeable, and therefore not acceptable. This means you need some way to make sure functions that don't ask for ordered kwargs don't get it (unless you can actually make it not noticeably slower). I think all of your solutions (including storing __kworder__) will be noticeably slower; there's just no way around the fact that maintaining a dict and its order and iterating it in custom order takes longer than maintaining a dict and iterating it in natural order. Trying to disguise the ordering info, or make it not quite as slow, or make it opt-out instead of opt-in, etc., doesn't solve this problem. Only finding a way to do nothing at all when nobody wants it solves this problem. At which point you might as well do the simplest thing—use OrderedDict—for cases where people _do_ want it. And that comes back to my original point: Where does someone want order? When they explicitly ask for it, but also when wrapping a function that asks for it in a generic wrapper. And that's the hard part; inside the wrapper (in some library code), you don't know to ask for it; outside the wrapper (in your app), there's no way to do it. Meanwhile, the "uncommon" cases (which you elsewhere call "rare", "extreme", and "edge cases") are a different story. I don't think Guido thinks they're very uncommon. He says, "I write code regularly that…" saves kwargs for later, and possibly even piles other stuff into it. His primary issue here is that having anything other than a plain dict is wrong and confusing. Secondarily, he also mentions "possible" performance problems. Here is where the memory comes in. Increasing the memory of every kwargs dict by 25% instead of 75% is nice, but I don't think it solves the memory problem, and I don't think the memory problem is the main issue anyway. This seems to rule out any solution that gives you a modified kwargs dict when you don't ask for it, whether it's an OrderedDict or a dict with an __order__ attribute or anything else. I now understand why you proposed the separate __kworder__ magic parameter solution. But really, given the first problem, any solution that adds _any_ noticeable overhead when you don't ask for it is already ruled out, so trying to put the extra overhead somewhere other than the dict itself doesn't help. You need to actually not create it. Anyway, after a little more back-burner thought, I actually did think of some magic that might work. But I don't think you, or anyone else, will like it. I certainly don't. At compile time, any code object that has a call to something accessed via a closure cell gets a calls-local flag. (This should catch all decorators and other wrapper-generators, but many false positives as well.) At runtime, when building a function from a code object with the calls-local flag, if any of the __closure__ members are functions with the order-kwargs flag, the new function also gets the flag as well. (This eliminates most of the false positives, without eliminating any actual wrappers unless they're designed pathologically. And if they are pathological, well, then they lose kwargs ordering.) That's all only a couple lines of code in the compiler and interpreter. Tada? Not sure; the idea is too horrible and non-Pythonic to think through all the way. Ultimately, this design would mean that all local functions get ordered kwargs, but there's an optimization that usually eliminates the order when you don't need it… which is just wrong.
On Apr 3, 2014 11:42 PM, "Andrew Barnert" <abarnert@yahoo.com> wrote:
From: Eric Snow <ericsnowcurrently@gmail.com>
Sent: Thursday, April 3, 2014 8:03 PM
On Thu, Apr 3, 2014 at 7:50 PM, Andrew Barnert <abarnert@yahoo.com>
wrote:
[snip]
So you are saying that it is not uncommon to use "perfect passthrough" decorators/wrappers you do not control.
Yes. Just my first example, partial, is ridiculously common, and it's a
wrapper I don't control. Half the servers, GUIs, and other callback-based programs I've worked on need partial functions, and get them either by using partial, or by using lambda functions instead--which has an equivalent and parallel problem that, come to think of it, still requires a solution too...
However, what we _don't_ know is how common it would be to use such
wrappers with functions that need ordered kwargs. The prototypical case of using partial on GUI or server callbacks obviously doesn't apply there--in a Tkinter GUI, your callback functions almost never take kwargs, and, if they did, it would probably be just to pass the dict as a widget configure dict or something like that, which couldn't possibly care about the order. But really, _no_ example that works today could apply; it's not possible to write functions that care about their keyword order, so we can only try to guess whether the potential use cases for them could involve wrapping them or not.
Regardless, the workarounds are the same. Likewise they add the same complication. Perhaps it does favor the decorator syntax over the __kworder__ local variable.
I'm not sure it favors either. Either way, every function that returns a
forwarding wrapper will need to be changed to apply the workaround. The workaround itself may be easier, or at least prettier, with the decorator, but the hard part isn't the workaround itself (as I showed, it could just be a simple on-line change), but figuring out which wrappers need it, and possibly copying and forking them into your own code, so you can apply the workaround to each one. Right. That's why a ordered-by-default approach is favorable. Providing an opt-out mechanism should satisfy what I still think are uncommon cases.
[snip]
Looking into the implications on inspect.signature, et al. is on my to-do list. :)
After a little more thought, what's really needed is a consistent way to
represent ordered-kwargs parameters in *documentation*.
Once you have that, whatever it is, inspect.signature should be simple.
For example, if you use kind=VAR_ORDERED_KEYWORD, then it's just a matter of making Parameter.__str__ return whatever-it-is that the docs would have instead of returning the normal **{name}.
This does give a little more weight to proposals that do something
visible in the signature itself--the ***okwargs, or an annotation, or whatever. Makes sense.
But I think all of them have enough down-sides compared to your more favored versions that it wouldn't be nearly enough weight to shift things.
Guido's concern was that in some uncommon cases the difference in performance (both memory and speed) would render OrderedDict undesirable for **kwargs. By storing the initial order on a dict attribute the only overhead would be 1 pointer per dict plus the size of the container.
First, after reading the messages you linked, I don't think your characterizing Guido's concerns fairly.
His first message is not about uncommon cases at all, it's about the fact
It's hard to predict how much of a slowdown a nice C OrderedDict would cause here, and how much less of a slowdown just creating and using __order__ would cause. But from a quick simulation in pure Python, it looks
Perhaps. At this point nothing much has been settled. My gut says you're right, but we'll see. that every function call (at least every one that uses kwargs) will slow down noticeably: "My main concern is speed -- since most code doesn't need it and function calls are already slow (and obviously very common :-) it would be a shame if this slowed down function calls that don't need it noticeably." In the best case the slowdown on each call would be the cost of checking a flag and not even for most calls. In the worst case I image it will be the cost of allocating a tuple/list (a free list would help) and populating it, while again this would only impact functions that define **kwargs). like you're just turning a 2.7x slowdown into a 2.2x slowdown--better, but still clearly noticeable, and therefore not acceptable. At one point I benchmarked my C OrderedDict (issue #16991) against dict. For most operations it was the same, including all non-mutating operations. If I recall correctly, iteration was even faster (due to traversing a linked list rather than a sparse hash table). For mutating operations the overhead resulted in up to a 3x slowdown.
This means you need some way to make sure functions that don't ask for
I think all of your solutions (including storing __kworder__) will be noticeably slower; there's just no way around the fact that maintaining a dict and its order and iterating it in custom order takes longer than
Trying to disguise the ordering info, or make it not quite as slow, or make it opt-out instead of opt-in, etc., doesn't solve this problem. Only finding a way to do nothing at all when nobody wants it solves this
ordered kwargs don't get it (unless you can actually make it not noticeably slower). The fact that this only affects functions that define **kwargs minimizes the scope of the impact. maintaining a dict and iterating it in natural order. I'm pretty sure this only matters in the **kwargs unpacking case. Even then, if we go with OrderedDict unpacking into the new kwargs is only marginally slower than dict. I do not know the impact of unpacking into a secondary order-preserving container in addition to into dict, but I expect it will not be much worse. Again this should only impact cases where the function defines **kwargs. problem. At which point you might as well do the simplest thing--use OrderedDict--for cases where people _do_ want it.
And that comes back to my original point: Where does someone want order?
When they explicitly ask for it, but also when wrapping a function that asks for it in a generic wrapper. And that's the hard part; inside the wrapper (in some library code), you don't know to ask for it; outside the wrapper (in your app), there's no way to do it. I agree that this use case ("perfect kwargs preserving" wrappers) is more complicated under the circumstances you described. However, don't discount the options on the table that do not present those circumstances.
Meanwhile, the "uncommon" cases (which you elsewhere call "rare",
"extreme", and "edge cases") are a different story. I don't think Guido thinks they're very uncommon. He says, "I write code regularly that..." saves kwargs for later, and possibly even piles other stuff into it. And I will argue that while perhaps not uncommon for Guido, they are uncommon for most people.
His primary issue here is that having anything other than a plain dict is wrong and confusing.
Hmm. I did not get that, much less that it is his primary concern. I took away that he was concerned about the performance impact in the case where kwargs outlives the function. Plus, this is Python. As long as it quacks like a dict we're fine. With OrderedDict it's even a subclass of dict.
Secondarily, he also mentions "possible" performance problems. Here is where the memory comes in. Increasing the memory of every kwargs dict by 25% instead of 75% is nice, but I don't think it solves the memory problem, and I don't think the memory problem is the main issue anyway.
I agree it's not the main issue.
This seems to rule out any solution that gives you a modified kwargs dict
I now understand why you proposed the separate __kworder__ magic
when you don't ask for it, whether it's an OrderedDict or a dict with an __order__ attribute or anything else. Due the memory issues? Memory is only an issue in cases where the resultant kwargs is going to be extremely large, which I still say is uncommon. Furthermore, using dict.__order__ means that even then it's an issue only with an extremely large number of kwargs passed in since __order__ wouldn't grow with the dict. And opt-out would allow people to avoid the issue in those cases. parameter solution. But really, given the first problem, any solution that adds _any_ noticeable overhead when you don't ask for it is already ruled out, so trying to put the extra overhead somewhere other than the dict itself doesn't help. You need to actually not create it. There shouldn't be any noticeable overhead so this point is somewhat moot. If there is then this proposal is on pretty shaky ground.
Anyway, after a little more back-burner thought, I actually did think of
At compile time, any code object that has a call to something accessed via a closure cell gets a calls-local flag. (This should catch all decorators and other wrapper-generators, but many false positives as well.) At runtime, when building a function from a code object with the calls-local flag, if any of the __closure__ members are functions with the order-kwargs flag, the new function also gets the flag as well. (This eliminates most of the false positives, without eliminating any actual wrappers unless they're designed pathologically. And if they are
design would mean that all local functions get ordered kwargs, but
some magic that might work. But I don't think you, or anyone else, will like it. I certainly don't. It's good to look at all the options. And this is python-ideas after all. :) pathological, well, then they lose kwargs ordering.) That's all only a couple lines of code in the compiler and interpreter. Tada? Not sure; the idea is too horrible and non-Pythonic to think through all the way. Ultimately, this there's an optimization that usually eliminates the order when you don't need it... which is just wrong. That's not so bad. I don't think something like that would be needed though. This is definitely an implementation detail. It would be important to minimize any further complication to function-call code. Otherwise the implementation will be a hard sell. Sort of relatedly, one thing to consider when adding language features is the impact they will have on other implementations. That relates almost entirely on the explicit additions to the language spec so we need to be careful about favoring solutions strictly for implementation concerns. -eric
This proposal strikes me as conceptually disordered. A set is not ordered. An ordered set would be a sequence, and if you want a sequence, the best thing is to use a sequence object. A dict is a set of key,value pairs. If you want an ordered sequence of key, value pairs, use a sequence. OrderedDicts were added for situations where that is not possible because a pre-existing api requires a mapping object, even though you might prefer to use a sequence. Passing a sequence of key,value pairs is a case that requires a mapping. If a function attends to the order of key,value pairs it receives, it should receive a sequence of such. **kwds in a call is a substitute for writing out unordered key=value specifications. The function may or may not have **kargs in its signature and if it does, the kargs received may or may not be equal to the kwds passed. -- Terry Jan Reedy
On Fri, Apr 4, 2014, at 4:54, Terry Reedy wrote:
**kwds in a call is a substitute for writing out unordered key=value specifications. The function may or may not have **kargs in its signature and if it does, the kargs received may or may not be equal to the kwds passed.
The problem is, there is no such thing as "writing out unordered key=value specifications" - anything that is "written out" has an order by its nature; the fact that this order is lost immediately is a poor abstraction. The desired use case is not to preserve the order of **kwds, it's to preserve the order of a sequence of key=value specifications.
I confess that after reading this thread, and a number of related past ones, I'm still not certain exactly what problem all of this is needed to solve. ISTM that if one has a special function calling requirement to pass in an ordered collection of key/value pairs, one can already just use a special and available call signature for your function: def myfunc(a, b, *keyvals): od = OrderedDict(keyvals) # ... do other stuff Call this like: value = myfunc(foo, bar, ('a',1), ('z',2), ('b',3)) Yes, it's a slightly special form of the calling convention, but it does something slightly special with its key/val-like arguments, so that seems like a reasonable tradeoff. The solution is purely local to the writer of the function who needs this. Even if you have an existing OrderedDict that you want to pass in, you can use that like: value = myfunc(foo, bar, *myOD.items()) Of course, if you want to be picky, you could stick in a check at the top of your function definition: assert all(isinstance(x, tuple) and len(x)==2 for x in keyvals) -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Fri, Apr 4, 2014, at 12:04, David Mertz wrote:
I confess that after reading this thread, and a number of related past ones, I'm still not certain exactly what problem all of this is needed to solve. ISTM that if one has a special function calling requirement to pass in an ordered collection of key/value pairs, one can already just use a special and available call signature for your function:
People want to be able to pass them in order __with the a=1, b=2 syntax__. Because the syntax you're proposing looks like grit on my monitor. The most commonly cited desired use case is for the OrderedDict constructor itself.
On Apr 4, 2014, at 10:15, random832@fastmail.us wrote:
On Fri, Apr 4, 2014, at 12:04, David Mertz wrote:
I confess that after reading this thread, and a number of related past ones, I'm still not certain exactly what problem all of this is needed to solve. ISTM that if one has a special function calling requirement to pass in an ordered collection of key/value pairs, one can already just use a special and available call signature for your function:
People want to be able to pass them in order __with the a=1, b=2 syntax__. Because the syntax you're proposing looks like grit on my monitor.
The most commonly cited desired use case is for the OrderedDict
Every time this comes up, that's the only good example anyone comes up with. It's obviously a special case. Besides being self-referential, a solution to that one case would automatically drastically improve the workarounds for any other case that anyone else later finds. If we need a general-purpose solution, then it has to be general-purpose. It has to work with dynamically-identified callables, forwarding wrappers, etc. And that's hard. If we only need OrderedDict to work, there are a lot fewer requirements.
On Apr 4, 2014, at 9:04, David Mertz <mertz@gnosis.cx> wrote:
def myfunc(a, b, *keyvals): od = OrderedDict(keyvals) # ... do other stuff
Call this like:
value = myfunc(foo, bar, ('a',1), ('z',2), ('b',3))
Or, possibly more readably: def myfunc(a, b, *keydicts): od = OrderedDict((k, v) for keydicts in keydicts for k, v in keydict.items()) value = myfunc(foo, bar, {'a': 1}, {'z': 2}) Not as clean as passing a single dict literal, or a sequence of keywords, but still clearly a sequence of key-value pairs rather than just a sequence of sequences...
On Apr 4, 2014 10:04 AM, "David Mertz" <mertz@gnosis.cx> wrote:
I confess that after reading this thread, and a number of related past ones, I'm still not certain exactly what problem all of this is needed to solve.
ISTM that if one has a special function calling requirement to pass in an ordered collection of key/value pairs, one can already just use a special and available call signature for your function:
def myfunc(a, b, *keyvals): od = OrderedDict(keyvals) # ... do other stuff
Call this like:
value = myfunc(foo, bar, ('a',1), ('z',2), ('b',3))
Yes, it's a slightly special form of the calling convention, but it does something slightly special with its key/val-like arguments, so that seems
Yeah, it won't be worth it without justification. :) I'll spell it out in the PEP, but basically it boils down to these reasons: * Helpful in debugging (from Raymond and others). * Controlling object presentation. * Anywhere you want to set iteration order and name/value in a single call (from Nick). + Factories for ordered types. + Serializable objects where order matters. * Keyword argument priority. I have a few more less concrete use cases as well. OrderedDict is the first example everyone cites, but it's simply the most obvious. Furthermore, it's a rather self-referential example. :) As I said, I'll make the full case in the PEP, which I hope to get out today. like a reasonable tradeoff. The solution is purely local to the writer of the function who needs this. That's clever. However, it is basically the same as the status quo: if you want to preserve order you cannot use keyword arguments, including ** unpacking, even though they are the natural way to do it. For existing functions, preserving order means changing code even though the order is already available. The point of my proposal is to teach the interpreter to preserve the order. -eric
On 6 Apr 2014 05:12, "Eric Snow" <ericsnowcurrently@gmail.com> wrote:
On Apr 4, 2014 10:04 AM, "David Mertz" <mertz@gnosis.cx> wrote:
I confess that after reading this thread, and a number of related past
ones, I'm still not certain exactly what problem all of this is needed to solve.
Yeah, it won't be worth it without justification. :) I'll spell it out
in the PEP, but basically it boils down to these reasons:
* Helpful in debugging (from Raymond and others). * Controlling object presentation. * Anywhere you want to set iteration order and name/value in a single
call (from Nick).
+ Factories for ordered types. + Serializable objects where order matters.
* Keyword argument priority.
I have a few more less concrete use cases as well. OrderedDict is the first example everyone cites, but it's simply the most obvious. Furthermore, it's a rather self-referential example. :) As I said, I'll make the full case in the PEP, which I hope to get out today.
ISTM that if one has a special function calling requirement to pass in an ordered collection of key/value pairs, one can already just use a special and available call signature for your function:
def myfunc(a, b, *keyvals): od = OrderedDict(keyvals) # ... do other stuff
Call this like:
value = myfunc(foo, bar, ('a',1), ('z',2), ('b',3))
Yes, it's a slightly special form of the calling convention, but it does something slightly special with its key/val-like arguments, so that seems like a reasonable tradeoff. The solution is purely local to the writer of the function who needs this.
That's clever. However, it is basically the same as the status quo: if you want to preserve order you cannot use keyword arguments, including ** unpacking, even though they are the natural way to do it. For existing functions, preserving order means changing code even though the order is already available. The point of my proposal is to teach the interpreter to
In the context of serialisation, one key lesson we have learned is that arbitrary ordering is a problem when you want to minimise spurious diffs, and sorting isn't a simple solution. Tools like doctest don't tolerate spurious diffs at all, but are often amenable to a sorting based answer. The cases where it would be highly desirable to be able use keyword arguments to control the order of display of a collection of key value pairs are ones like: * printing out key:value pairs in CLI output * mapping semantic names to column order in a CSV * serialising attributes and elements in particular orders in XML * serialising map keys in particular orders in human readable formats like JSON and YAML (particularly when they're going to be placed under source control) These *can* all be done today, but *not* by using keyword arguments. In my view, the problem to be addressed is that keyword arguments *look* like they should work for these cases, because they have a definite order in the source code. The only reason they don't work is because the interpreter throws that ordering information away. It's a textbook case of a language feature becoming an attractive nuisance in some circumstances: the simple and obvious solution for the above use cases *doesn't actually work* for reasons that aren't obviously clear if you don't have a firm grasp of Python's admittedly complicated argument handling. The simplest way out that I can see? Just make it work, even if that means also preserving the order of arbitrary keyword arguments in cases that *dont* need it. Deciding whether or not to provide a way to opt in to dropping the order info for speed and memory reasons then becomes a separate optimisation discussion *after* the current usability trap has been addressed. Cheers, Nick. preserve the order.
-eric
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Apr 5, 2014 3:38 PM, "Nick Coghlan" <ncoghlan@gmail.com> wrote:
In the context of serialisation, one key lesson we have learned is that arbitrary ordering is a problem when you want to minimise spurious diffs, and sorting isn't a simple solution.
Good point.
Tools like doctest don't tolerate spurious diffs at all, but are often
These *can* all be done today, but *not* by using keyword arguments. In my view, the problem to be addressed is that keyword arguments *look* like
amenable to a sorting based answer. [snip] they should work for these cases, because they have a definite order in the source code. The only reason they don't work is because the interpreter throws that ordering information away.
It's a textbook case of a language feature becoming an attractive
nuisance in some circumstances: the simple and obvious solution for the above use cases *doesn't actually work* for reasons that aren't obviously clear if you don't have a firm grasp of Python's admittedly complicated argument handling. That's a great way of putting it.
The simplest way out that I can see? Just make it work, even if that
means also preserving the order of arbitrary keyword arguments in cases that *dont* need it. So you're advocating simply using OrderedDict for kwargs? Ultimately that's my preference too. The other options are attempts at addressing concerns that have come up, particularly Guido's. Also, "just make it work" must include not having a meaningful impact on function call performance. :) That was the big objection to the original proposal(s) to just switch kwargs to OrderedDict.
Deciding whether or not to provide a way to opt in to dropping the order
info for speed and memory reasons then becomes a separate optimisation discussion *after* the current usability trap has been addressed. Sort of. The PEP certainly needs to address this point one way or the other. Unless Guido relaxes his objections, that is. :) Regardless, once I have a draft PEP out the next step will be to implement the OrderedDict swap and see how things look performance-wise. It may turn out that the impact is marginal, at which point the PEP won't have to say much. -eric
On 6 Apr 2014 08:23, "Eric Snow" <ericsnowcurrently@gmail.com> wrote:
On Apr 5, 2014 3:38 PM, "Nick Coghlan" <ncoghlan@gmail.com> wrote:
In the context of serialisation, one key lesson we have learned is that
arbitrary ordering is a problem when you want to minimise spurious diffs, and sorting isn't a simple solution.
Good point.
Tools like doctest don't tolerate spurious diffs at all, but are often
[snip]
These *can* all be done today, but *not* by using keyword arguments. In my view, the problem to be addressed is that keyword arguments *look* like
It's a textbook case of a language feature becoming an attractive
nuisance in some circumstances: the simple and obvious solution for the above use cases *doesn't actually work* for reasons that aren't obviously clear if you don't have a firm grasp of Python's admittedly complicated argument handling.
That's a great way of putting it.
The simplest way out that I can see? Just make it work, even if that
means also preserving the order of arbitrary keyword arguments in cases
amenable to a sorting based answer. they should work for these cases, because they have a definite order in the source code. The only reason they don't work is because the interpreter throws that ordering information away. that *dont* need it.
So you're advocating simply using OrderedDict for kwargs? Ultimately
Also, "just make it work" must include not having a meaningful impact on function call performance. :) That was the big objection to the original
that's my preference too. The other options are attempts at addressing concerns that have come up, particularly Guido's. In my experience, the best way to go with a PEP is to make the best case you can for the solution you *really* want, and then let others make the case for why that isn't possible or acceptable. Adjusting the PEP accordingly as people persuade you to change your mind then helps ensure that the rationale for various design decisions is accurately recorded in the final PEP that is submitted for pronouncement. Acknowledge the known objections up front, and try to make a persuasive case to counter them. In this case, actually making the change and checking the impact on the macro and micro benchmark suites is likely the best way to bring actual data to bear on the question of the performance implications. The initial draft doesn't need to include that data, just a promise to collect it before submitting the PEP for a final decision (see Christian's hash PEP for an example of a "speed impact is likely real, but within acceptable limits given the other gains" proposal). proposal(s) to just switch kwargs to OrderedDict. Yep, but performance concerns are notorious for needing to be measured to see the true impact. A lot of speed critical code already uses positional args instead of keyword args, and APIs like str.format_map exist specifically to bypass the whole kwarg dance entirely. So it isn't *at all* clear how much overall impact this change will have outside micro benchmarks. It may also be possible to micro-optimise the "only 1 arbitrary kwarg" case to use an ordinary dict (if benchmarking backs up the stated concern that the simpler solution of always using odict may be too slow or use too much memory and handling cases like "key" arguments more efficiently then turns out to help address that). But we'll never know if we pre-emptively concede to the speculative concerns about the performance impact, rather than attempting to gather the data needed to show whether or not the simplest, most comprehensive solution can be attained with a negligible (or at least acceptable) speed impact. Cheers, Nick.
Deciding whether or not to provide a way to opt in to dropping the
order info for speed and memory reasons then becomes a separate optimisation discussion *after* the current usability trap has been addressed.
Sort of. The PEP certainly needs to address this point one way or the
other. Unless Guido relaxes his objections, that is. :) Regardless, once I have a draft PEP out the next step will be to implement the OrderedDict swap and see how things look performance-wise. It may turn out that the impact is marginal, at which point the PEP won't have to say much.
-eric
On 4/4/2014 9:50 AM, random832@fastmail.us wrote:
On Fri, Apr 4, 2014, at 4:54, Terry Reedy wrote:
**kwds in a call is a substitute for writing out unordered key=value specifications. The function may or may not have **kargs in its signature and if it does, the kargs received may or may not be equal to the kwds passed.
The problem is, there is no such thing as "writing out unordered key=value specifications" - anything that is "written out" has an order by its nature;
Before being written, the key=value specification is unordered. The order introduced by writing it out in linear text is spurious.
the fact that this order is lost immediately is a poor abstraction. The desired use case is not to preserve the order of **kwds, it's to preserve the order of a sequence of key=value specifications.
Wanting to preserve spurious order is wrong. What I said before is that if one has meaningful order, one should better use an ordered sequence, both in the definition and call of the function, rather than an unordered (or spuriously ordered) mapping. -- Terry Jan Reedy
On Apr 4, 2014 9:17 PM, "Terry Reedy" <tjreedy@udel.edu> wrote:
Before being written, the key=value specification is unordered. The order introduced by writing it out in linear text is spurious. [snip] Wanting to preserve spurious order is wrong. What I said before is that if one has meaningful order, one should better use an ordered sequence, both in the definition and call of the function, rather than an unordered (or spuriously ordered) mapping.
Sometimes the order is not spurious. That's the point of making sure we enumerate valid use cases for this proposal. Otherwise the cost to the language isn't worth it. -eric
On Apr 4, 2014 2:54 AM, "Terry Reedy" <tjreedy@udel.edu> wrote:
This proposal strikes me as conceptually disordered. A set is not
ordered. An ordered set would be a sequence, and if you want a sequence, the best thing is to use a sequence object. A dict is a set of key,value pairs. If you want an ordered sequence of key, value pairs, use a sequence. OrderedDicts were added for situations where that is not possible because a pre-existing api requires a mapping object, even though you might prefer to use a sequence. With OrderedDict it's about having a mapping and the order of the keys in one data structure. A sequence of pairs does not an ordered mapping make, though they may contain the same data.
Passing a sequence of key,value pairs is a case that requires a mapping.
If a function attends to the order of key,value pairs it receives, it should receive a sequence of such.
**kwds in a call is a substitute for writing out unordered key=value
specifications. The function may or may not have **kargs in its signature and if it does, the kargs received may or may not be equal to the kwds passed. As someone else noted, you write out an *ordered* key=value specification and that information is lost. In the ** unpacking case, any order information in the unpacked object is likewise lost. This proposal is all about preserving that order information in some way without requiring the caller to extract the order information manually and passing it in separately. -eric
participants (6)
-
Andrew Barnert -
David Mertz -
Eric Snow -
Nick Coghlan -
random832＠fastmail.us -
Terry Reedy