PEP 612: Parameter Specification Variables Discussion Pt. II
Hi All, I want to start off by thanking everyone that gave feedback on the first round of review of PEP 612. Your feedback illuminated a lot of things that I think have made this PEP a lot better. Over the past few weeks I have had the time to rewrite much of that document to integrate all of that feedback. Notably, this included integrating `typing.type_variable_operators.Concatenate` to support decorators that modify a finite number of positional parameters. With that being said, I'd appreciate another round of feedback from all of you to determine whether this form of the PEP is ready to move on to the Steering Council. Thanks again! Mark Mendoza
Hi Mark, I really like this PEP - it'll be useful to have more expressive ways of annotating decorators. The current form looks pretty good to me. A few comments: * `typing.type_variable_operators` is quite verbose. How about just `typing.operators`? * For user-defined generic classes using a ParamSpec, I think it would be useful to include an example of how such a class is constructed, since it wasn't obvious to me where values with the parameters_expression type would come from at runtime. Is class X(Generic[P]): def __init__(self, x: Callable[P, int]): ... and variations the only way to create such a class? Or do P.args and P.kwargs come into play in some way? * "Inside the function, args has the type P.args, not Tuple[P.args, ...] as would be with a normal annotation (and likewise with the **kwargs)" (in the ParamSpec semantics <https://www.python.org/dev/peps/pep-0612/#id3> section). If I'm reading this correctly, it's saying that P.args represents the full type of args, Tuple and all, unlike a normal args annotation which describes just the contained type. Why is that the case? It seems like it would be better to not have this inconsistency unless it's necessary. Also, some typos and such I noticed: * "a*n* concrete instance" has an extraneous "n": https://github.com/python/peps/blame/master/pep-0612.rst#L231 * I found the x_int_y_str, y_int_x_str <https://github.com/python/peps/blame/master/pep-0612.rst#L283> example hard to read, since the slight differences between the two foo calls are hard to spot. One way I can think of to improve this would be to make the function names shorter (maybe just x_y and y_x?) so it takes less time to spot the difference. * Is this supposed to be Callable[[int, str], *int*]? https://github.com/python/peps/blame/master/pep-0612.rst#L291 * The parameter to the remove function is defined as `x` and later referred to as `f`: https://github.com/python/peps/blame/master/pep-0612.rst#L465 Best, Rebecca On Mon, Jul 13, 2020 at 8:47 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi All, I want to start off by thanking everyone that gave feedback on the first round of review of PEP 612. Your feedback illuminated a lot of things that I think have made this PEP a lot better. Over the past few weeks I have had the time to rewrite much of that document to integrate all of that feedback. Notably, this included integrating `typing.type_variable_operators.Concatenate` to support decorators that modify a finite number of positional parameters.
With that being said, I'd appreciate another round of feedback from all of you to determine whether this form of the PEP is ready to move on to the Steering Council.
Thanks again!
Mark Mendoza _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: rechen@google.com
On Thu, Jul 30, 2020 at 7:27 PM Rebecca Chen via Typing-sig < typing-sig@python.org> wrote:
Hi Mark,
I really like this PEP - it'll be useful to have more expressive ways of annotating decorators. The current form looks pretty good to me. A few comments:
* `typing.type_variable_operators` is quite verbose. How about just `typing.operators`?
Hm, why a submodule at all?
* For user-defined generic classes using a ParamSpec, I think it would be useful to include an example of how such a class is constructed, since it wasn't obvious to me where values with the parameters_expression type would come from at runtime. Is
class X(Generic[P]): def __init__(self, x: Callable[P, int]): ...
and variations the only way to create such a class? Or do P.args and P.kwargs come into play in some way?
Yes, this would be helpful to readers. * "Inside the function, args has the type P.args, not Tuple[P.args, ...] as
would be with a normal annotation (and likewise with the **kwargs)" (in the ParamSpec semantics <https://www.python.org/dev/peps/pep-0612/#id3> section). If I'm reading this correctly, it's saying that P.args represents the full type of args, Tuple and all, unlike a normal args annotation which describes just the contained type. Why is that the case? It seems like it would be better to not have this inconsistency unless it's necessary.
It is indeed an inconsistency. But how else would you spell it? In PEP 484, the type of `*args` and `**kwargs` is *always* the type for an individual item (so without the tuple or dict). But `P.args` and `P.kwargs` doesn't have a single item type. If we could make up new syntax, we could write e.g. `f(*args: *P.args, **kwargs: *P.kwargs)` but I really don't think we should try to invent new syntax -- it would make it infeasible to use the new feature before Python 3.10. Essentially `P.args` and `P.kwargs` are magic cookies -- the type checker will have to special-case these no matter what. (And these cannot be used in any other context, the PEP makes clear.)
Also, some typos and such I noticed:
* "a*n* concrete instance" has an extraneous "n": https://github.com/python/peps/blame/master/pep-0612.rst#L231 * I found the x_int_y_str, y_int_x_str <https://github.com/python/peps/blame/master/pep-0612.rst#L283> example hard to read, since the slight differences between the two foo calls are hard to spot. One way I can think of to improve this would be to make the function names shorter (maybe just x_y and y_x?) so it takes less time to spot the difference. * Is this supposed to be Callable[[int, str], *int*]? https://github.com/python/peps/blame/master/pep-0612.rst#L291 * The parameter to the remove function is defined as `x` and later referred to as `f`: https://github.com/python/peps/blame/master/pep-0612.rst#L465
Thanks! I fixed the first and the last of these (and two I caught myself) and quietly pushed the correction. The 2nd and 3rd I'll leave for Mark to fix. And FWIW, once we are clear on the final few issues, I intend to recommend this PEP for approval by the SC. (In fact, I think the only contentious issue is the namespace where Concatenate should live.) --Guido
Best, Rebecca
On Mon, Jul 13, 2020 at 8:47 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi All, I want to start off by thanking everyone that gave feedback on the first round of review of PEP 612. Your feedback illuminated a lot of things that I think have made this PEP a lot better. Over the past few weeks I have had the time to rewrite much of that document to integrate all of that feedback. Notably, this included integrating `typing.type_variable_operators.Concatenate` to support decorators that modify a finite number of positional parameters.
With that being said, I'd appreciate another round of feedback from all of you to determine whether this form of the PEP is ready to move on to the Steering Council.
Thanks again!
Mark Mendoza _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: rechen@google.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Hi Rebecca & Guido,
Hm, why a submodule at all?
I opted for a submodule in anticipation of future operators such as `Map`, which I thought might be confusing if were just at the top-level of `typing`. I don't have strong opinions on this, and am happy to go with either `typing.operators.Concatenate` or just `typing.Concatenate`.
It is indeed an inconsistency. But how else would you spell it?
I totally agree with this explanation. Rebecca, do you think there's a way to modify the PEP to make this aspect less confusing?
I think it would be useful to include an example of how such a class is constructed,
The 2nd and 3rd I'll leave for Mark to fix.
I added the requested example and fixed the two typos in this PR: https://github.com/python/peps/pull/1545 Thanks for the feedback! Best, Mark
On Fri, Jul 31, 2020 at 12:09 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi Rebecca & Guido,
Hm, why a submodule at all?
I opted for a submodule in anticipation of future operators such as `Map`, which I thought might be confusing if were just at the top-level of `typing`. I don't have strong opinions on this, and am happy to go with either `typing.operators.Concatenate` or just `typing.Concatenate`.
My concern was mainly that a long submodule name would be tedious to use when importing, etc., so either typing.operators.Concatenate or typing.Concatenate seems reasonable to me. Personally, I would slightly lean towards having the submodule if there are likely to be future operators.
It is indeed an inconsistency. But how else would you spell it?
I totally agree with this explanation. Rebecca, do you think there's a way to modify the PEP to make this aspect less confusing?
I had to read the explanation a few times, but I think I get it now: P.args and P.kwargs don't always refer to the same subsets of the arguments, so there's no fixed contained type they could represent. Maybe to be helpful, the PEP could briefly explain this? "*Because [reason],* args has the type P.args inside the function..."
I think it would be useful to include an example of how such a class is constructed,
The 2nd and 3rd I'll leave for Mark to fix.
I added the requested example and fixed the two typos in this PR: https://github.com/python/peps/pull/1545
Thanks for the feedback!
Best, Mark _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: rechen@google.com
Hi Rebecca, Sounds good! I have made both of your requested changes in this PR: https://github.com/python/peps/pull/1546 Best, Mark
On Fri, Jul 31, 2020 at 3:48 PM Rebecca Chen via Typing-sig < typing-sig@python.org> wrote:
On Fri, Jul 31, 2020 at 12:09 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi Rebecca & Guido,
Hm, why a submodule at all?
I opted for a submodule in anticipation of future operators such as `Map`, which I thought might be confusing if were just at the top-level of `typing`. I don't have strong opinions on this, and am happy to go with either `typing.operators.Concatenate` or just `typing.Concatenate`.
My concern was mainly that a long submodule name would be tedious to use when importing, etc., so either typing.operators.Concatenate or typing.Concatenate seems reasonable to me. Personally, I would slightly lean towards having the submodule if there are likely to be future operators.
I want to push back here some more. The typing operators are only useful with the types, and the types are not in a sub-namespace. Even if there were a future operator `Map`, it doesn't actually clash with the `Mapping` ABC, and most people are just going to write `from typing.operators import *`. And for those people who prefer to do something like `import typing as t` and then write `t.Any`, `t.TypeVar` etc., `t.operators.Concatenate` would still be rather unwieldy. (I observe that the examples in the PEP use `from typing.operators import Concatenate`. :-) I added the requested example and fixed the two typos in this PR:
I'll hold off on that until we've agreed on where `Concatenate` will live. PS. Personally I would probably end up writing `from typing import Concatenate as Cat`, but that's just me being an old UNIX hack. :-) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Hi Guido, That sounds reasonable, and if we're comfortable with typing.Map, then I'm happy to make it `typing.Concatenate`. Just updated the PR with this. Best, Mark
The Steering Council has appointed me PEP Delegate for PEP 612. With the last bit of feedback out of the way I intend to accept the current draft, unless objections are made here within the next 24 hours. Thanks Mark for your work on the design and (IIUC) the prototype and for carefully taking the feedback offered into account. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
This isn't a serious objection, but I wanted to note that "concatenate" is an overloaded term, so I can imagine the name potentially being a point of confusion if it is intended only to refer specifically to concatenating parameters. In particular, if we/when we add typing for multi-dimensional array shapes, it would be very natural to talk about "concatenating" two array shapes, e.g., for a function that removes a leading axis from arrays: D = DimensionVar() S = ShapeVar() def sum_along_first_axis(array: NDArray[Concatenate[D, S]]) -> NDArray[S]: return array.sum(axis=0) This looks very similar to the usage for function parameters, but is manipulating a different type of object. On Mon, Aug 10, 2020 at 12:03 PM Guido van Rossum <guido@python.org> wrote:
The Steering Council has appointed me PEP Delegate for PEP 612.
With the last bit of feedback out of the way I intend to accept the current draft, unless objections are made here within the next 24 hours.
Thanks Mark for your work on the design and (IIUC) the prototype and for carefully taking the feedback offered into account.
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: shoyer@gmail.com
Hi Stephan, In Pyre's existing support for variadic type variables, (e.g. what you're spelling as ShapeVar), we do in fact re-use the Concatenate operator for that exact purpose. We haven't had any issues with using that operator for both purposes. Best, Mark Mendoza
On Mon, Aug 10, 2020 at 8:59 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi Stephan, In Pyre's existing support for variadic type variables, (e.g. what you're spelling as ShapeVar), we do in fact re-use the Concatenate operator for that exact purpose. We haven't had any issues with using that operator for both purposes. Best, Mark Mendoza
Sounds great then! It might be worth clarifying somewhere that the intended usage for "Concatenate" is not necessarily restricted to only function parameters. Or perhaps this discussion is enough documentation :).
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: shoyer@gmail.com
I have now accepted PEP 612. Congrats again Mark! Regarding the clarification of Concatenate, I think it should be okay to leave that as is for now -- when a new PEP is written to standardize the other usage it will be up to that PEP to explain that there is no ambiguity there. --Guido On Mon, Aug 10, 2020 at 9:23 PM Stephan Hoyer <shoyer@gmail.com> wrote:
On Mon, Aug 10, 2020 at 8:59 PM Mark Mendoza <mendoza.mark.a@gmail.com> wrote:
Hi Stephan, In Pyre's existing support for variadic type variables, (e.g. what you're spelling as ShapeVar), we do in fact re-use the Concatenate operator for that exact purpose. We haven't had any issues with using that operator for both purposes. Best, Mark Mendoza
Sounds great then!
It might be worth clarifying somewhere that the intended usage for "Concatenate" is not necessarily restricted to only function parameters. Or perhaps this discussion is enough documentation :).
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: shoyer@gmail.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
participants (4)
-
Guido van Rossum -
Mark Mendoza -
Rebecca Chen -
Stephan Hoyer