Tensor Typing presentation and discussion on Wednesday! (As part of the NumPy Community Meeting)
Hi all, There will be a NumPy Community meeting Wednesday June 9th at 20:00 UTC. This meeting will be dedicated to a presentation by Matthew and Pradeep about typing arrays. The title is: "Better types for numerical computing - arrays generic in dtype and shape" Abstract: An overview of why more detailed types for arrays would be awesome, the work being done to enable those types in PEP 646, and what the future of this effort could look like. If you are interested in typing and/or missed out on the Python typing summit, this will be a perfect opportunity to join and discuss better typing of NumPy arrays! Details on how to join can be found in the usual meeting notes: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian
Hello! Any chance this could be recorded for those of us in useless time zones? š On Mon, 7 Jun 2021, at 6:03 PM, Sebastian Berg wrote:
Hi all,
There will be a NumPy Community meeting Wednesday June 9th at 20:00 UTC.
This meeting will be dedicated to a presentation by Matthew and Pradeep about typing arrays.
The title is: "Better types for numerical computing - arrays generic in dtype and shape"
Abstract: An overview of why more detailed types for arrays would be awesome, the work being done to enable those types in PEP 646, and what the future of this effort could look like.
If you are interested in typing and/or missed out on the Python typing summit, this will be a perfect opportunity to join and discuss better typing of NumPy arrays!
Details on how to join can be found in the usual meeting notes:
https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both
Best wishes
Sebastian
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Tue, 2021-06-08 at 11:40 -0700, Stefan van der Walt wrote:
On Tue, Jun 8, 2021, at 03:23, Juan Nunez-Iglesias wrote:
Hello! Any chance this could be recorded for those of us in useless time zones? š
I added a note to the agenda; we'll ask the speakers and record if they don't mind.
We have made a recording, but I did not plan on making it available publicly. Please contact me directly if you are interested. Cheers, Sebastian
StƩfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Some links I promised to send around: - https://github.com/deepmind/tensor_annotations which has a bunch of experimental shape-aware stubs for TensorFlow and JAX using 'semantic' axis labels - https://github.com/pradeep90/pytorch_examples ditto for TensorFlow and PyTorch using value-based axis labels - https://arxiv.org/pdf/2102.13254.pdf Similar work in TensorFlow for Swift - Slides <https://docs.google.com/presentation/d/1kLS-bz1ZmJIFizZExkm8N9jqAj2oOJzTmUaL8INGefc/edit?usp=sharing&resourcekey=0-eFDyGvcEeeHzF62x6Cwl6Q> for our presentation - Tensor typing meeting details <https://docs.google.com/document/d/1oaG0V2ZE5BRDjd9N-Tr1N0IKGwZQcraIlZ0N8ayqVg8/edit> Thanks again, everyone! On Wed, 9 Jun 2021 at 22:28, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Tue, 2021-06-08 at 11:40 -0700, Stefan van der Walt wrote:
On Tue, Jun 8, 2021, at 03:23, Juan Nunez-Iglesias wrote:
Hello! Any chance this could be recorded for those of us in useless time zones? š
I added a note to the agenda; we'll ask the speakers and record if they don't mind.
We have made a recording, but I did not plan on making it available publicly. Please contact me directly if you are interested.
Cheers,
Sebastian
StƩfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Thank you so much Matthew, Pradeep, and everyone else helping with this effort! And thanks Matthew for sharing the slides ā very informative! Iāve asked Sebastian for the recording but in the meantime I thought Iād ask a couple of questions about the syntax: * why isnāt there a Subtract operator? It seems that Subtract[A, B] is much more readable than Add[A, Multiply[Literal[-1], B]]? * is there a prospect for using actual operators and actual literals here? I think that a lot of reticence about Python typing comes from the annotations being so difficult to read. Thereās been some great improvements recently with āOptional[Tuple[Union[Float, Int]]]ā becoming ātuple[float | int] | Noneā, and I hope we can leapfrog the awkward phase with this tensor typing stuff. Again, A - B is much more readable again than Subtract[A, B]. And B-2 is much more readable than Add[B, Literal[-2]]. Other than that, love the work, thank you again! Juan.
On 11 Jun 2021, at 5:08 am, Matthew Rahtz <mrahtz@google.com> wrote:
Some links I promised to send around: https://github.com/deepmind/tensor_annotations <https://github.com/deepmind/tensor_annotations> which has a bunch of experimental shape-aware stubs for TensorFlow and JAX using 'semantic' axis labels https://github.com/pradeep90/pytorch_examples <https://github.com/pradeep90/pytorch_examples> ditto for TensorFlow and PyTorch using value-based axis labels https://arxiv.org/pdf/2102.13254.pdf <https://arxiv.org/pdf/2102.13254.pdf> Similar work in TensorFlow for Swift Slides <https://docs.google.com/presentation/d/1kLS-bz1ZmJIFizZExkm8N9jqAj2oOJzTmUaL8INGefc/edit?usp=sharing&resourcekey=0-eFDyGvcEeeHzF62x6Cwl6Q> for our presentation Tensor typing meeting details <https://docs.google.com/document/d/1oaG0V2ZE5BRDjd9N-Tr1N0IKGwZQcraIlZ0N8ayqVg8/edit> Thanks again, everyone!
On Wed, 9 Jun 2021 at 22:28, Sebastian Berg <sebastian@sipsolutions.net <mailto:sebastian@sipsolutions.net>> wrote: On Tue, 2021-06-08 at 11:40 -0700, Stefan van der Walt wrote:
On Tue, Jun 8, 2021, at 03:23, Juan Nunez-Iglesias wrote:
Hello! Any chance this could be recorded for those of us in useless time zones? š
I added a note to the agenda; we'll ask the speakers and record if they don't mind.
We have made a recording, but I did not plan on making it available publicly. Please contact me directly if you are interested.
Cheers,
Sebastian
StƩfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org <mailto:NumPy-Discussion@python.org> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org <mailto:NumPy-Discussion@python.org> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion> _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Hi Juan Thanks for the kind words! why isnāt there a Subtract operator?
I *think* this is just because the implementation in Pyre is still an early prototype. Alfonso? is there a prospect for using actual operators and actual literals here?
That's a really good question. We're optimistic that actual literals should be possible - we assumed there was some subtle reason we'd have to use Literal, but when we briefly talked about it at PyCon, Guido was like "Nope, you should just write a PEP for that". As for actual operators - I can't think of any *technical* reason this shouldn't be possible - when it comes time to write a PEP for type arithmetic it's definitely something we should propose. Alfonso, any thoughts? Best, Matthew On Mon, 14 Jun 2021 at 03:51, Juan Nunez-Iglesias <jni@fastmail.com> wrote:
Thank you so much Matthew, Pradeep, and everyone else helping with this effort!
And thanks Matthew for sharing the slides ā very informative! Iāve asked Sebastian for the recording but in the meantime I thought Iād ask a couple of questions about the syntax:
* why isnāt there a Subtract operator? It seems that Subtract[A, B] is much more readable than Add[A, Multiply[Literal[-1], B]]? * is there a prospect for using actual operators and actual literals here? I think that a lot of reticence about Python typing comes from the annotations being so difficult to read. Thereās been some great improvements recently with āOptional[Tuple[Union[Float, Int]]]ā becoming ātuple[float | int] | Noneā, and I hope we can leapfrog the awkward phase with this tensor typing stuff. Again, A - B is much more readable again than Subtract[A, B]. And B-2 is much more readable than Add[B, Literal[-2]].
Other than that, love the work, thank you again!
Juan.
On 11 Jun 2021, at 5:08 am, Matthew Rahtz <mrahtz@google.com> wrote:
Some links I promised to send around:
- https://github.com/deepmind/tensor_annotations which has a bunch of experimental shape-aware stubs for TensorFlow and JAX using 'semantic' axis labels - https://github.com/pradeep90/pytorch_examples ditto for TensorFlow and PyTorch using value-based axis labels - https://arxiv.org/pdf/2102.13254.pdf Similar work in TensorFlow for Swift - Slides <https://docs.google.com/presentation/d/1kLS-bz1ZmJIFizZExkm8N9jqAj2oOJzTmUaL8INGefc/edit?usp=sharing&resourcekey=0-eFDyGvcEeeHzF62x6Cwl6Q> for our presentation - Tensor typing meeting details <https://docs.google.com/document/d/1oaG0V2ZE5BRDjd9N-Tr1N0IKGwZQcraIlZ0N8ayqVg8/edit>
Thanks again, everyone!
On Wed, 9 Jun 2021 at 22:28, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Tue, 2021-06-08 at 11:40 -0700, Stefan van der Walt wrote:
On Tue, Jun 8, 2021, at 03:23, Juan Nunez-Iglesias wrote:
Hello! Any chance this could be recorded for those of us in useless time zones? š
I added a note to the agenda; we'll ask the speakers and record if they don't mind.
We have made a recording, but I did not plan on making it available publicly. Please contact me directly if you are interested.
Cheers,
Sebastian
StƩfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Hi Matthew & Juan, Thank you for the kind words and for the feedback! I totally agree in what you are suggesting, so I will just provide a little of context on why it was presented that way. First of all, when talking about type arithmetic I suggest to refer to the latest presentation for the summit rather than from the Tensor Typing Meetings (https://docs.google.com/presentation/d/1NZzZfbUCXi78LDDe8C3hAaK9on8ZiX3Z2k4B...). Here I intentionally refer to the +,-,*,// syntax rather than the Add[] one. Moreover, I also write 1 + A rather than Literal[1] + A, assuming that we could write a PEP for enabling this behavior if needed, and as Matthew said, it seems feasible based on what was said in the summit. So yes, we totally agree that is important to make the syntax as simple as possible, especially because expressions can get quite complicated (e.g. convolutions). Nonetheless, I will also answer the question of "Why in the presentation of the Tensor Typing meeting you used the Prefix operators (Add) and did not support directly Subtract?" The answer is that the goal was to show what was tecnically possible to achieve with Type Arithmetic, and syntax does not matter in that regard. Since we knew that the syntax could change any point later on we just went for the prefix one after a quick 30 minutes discussion in Pyre. However, the implementation for doing A+B instead of Add[A,B] would have been the same complex, and can be changed with a few lines of code. About Subtract, it was once again, because we did not need a custom Subtract operator to show what type arithmetic can do. However, supporting it would have been a matter of a few lines in Pyre, I even had a PR ready but we decided to make as few changes as possible since it would not add any new functionality. After all, the idea was how to bring a big feature like Type Arithmetic with as few changes to the language as possible. Best, Alfonso. Quoting Matthew Rahtz <mrahtz@google.com>:
Hi Juan Ā Thanks for the kind words! Ā
why isnāt there a Subtract operator?
Ā I /think/Ā this is just because the implementation in Pyre is still an early prototype. Alfonso? Ā
is there a prospect for using actual operators and actual literals here?
Ā That's a really good question. We're optimistic that actual literals should be possible - we assumed there was some subtle reason we'd have to use Literal, but when we briefly talked about it at PyCon, Guido was like "Nope, you should just write a PEP for that". As for actual operators - I can't think of any /technical/Ā reason this shouldn't be possible - when it comes time to write a PEP for type arithmetic it's definitely something we should propose. Alfonso, any thoughts? Ā Best, Matthew
On Mon, 14 Jun 2021 at 03:51, Juan Nunez-Iglesias <jni@fastmail.com> wrote:
Thank you so much Matthew, Pradeep, and everyone else helping with this effort! Ā And thanks Matthew for sharing the slides ā very informative! Iāve asked Sebastian for the recording but in the meantime I thought Iād ask a couple of questions about the syntax: Ā * why isnāt there a Subtract operator? It seems that Subtract[A, B] is much more readable than Add[A, Multiply[Literal[-1], B]]? * is there a prospect for using actual operators and actual literals here? I think that a lot of reticence about Python typing comes from the annotations being so difficult to read. Thereās been some great improvements recently with āOptional[Tuple[Union[Float, Int]]]ā becoming ātuple[float | int] | Noneā, and I hope we can leapfrog the awkward phase with this tensor typing stuff. Again, A - B is much more readable again than Subtract[A, B]. And B-2 is much more readable than Add[B, Literal[-2]]. Ā Other than that, love the work, thank you again! Ā Juan.
On 11 Jun 2021, at 5:08 am, Matthew Rahtz <mrahtz@google.com> wrote:
Some links I promised to send around:
* https://github.com/deepmind/tensor_annotations which has a bunch of experimental shape-aware stubs for TensorFlow and JAX using 'semantic' axis labels * https://github.com/pradeep90/pytorch_examples ditto for TensorFlow and PyTorch using value-based axis labels * https://arxiv.org/pdf/2102.13254.pdf Similar work in TensorFlow for Swift * Slides[1]Ā for our presentation * Tensor typing meeting details[2] Thanks again, everyone!
On Wed, 9 Jun 2021 at 22:28, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Tue, 2021-06-08 at 11:40 -0700, Stefan van der Walt wrote:
On Tue, Jun 8, 2021, at 03:23, Juan Nunez-Iglesias wrote:
Hello! Any chance this could be recorded for those of us in useless time zones? š
I added a note to the agenda; we'll ask the speakers and record if they don't mind.
We have made a recording, but I did not plan on making it available publicly.Ā Please contact me directly if you are interested.
Cheers,
Sebastian
StƩfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Links: ------ [1] https://docs.google.com/presentation/d/1kLS-bz1ZmJIFizZExkm8N9jqAj2oOJzTmUaL8INGefc/edit?usp=sharing&resourcekey=0-eFDyGvcEeeHzF62x6Cwl6Q [2] https://docs.google.com/document/d/1oaG0V2ZE5BRDjd9N-Tr1N0IKGwZQcraIlZ0N8ayq...
Hi Alfonso, Matthew, On Tue, Jun 15, 2021, at 06:59, ALFONSO LUIS CASTAĆO MARIN wrote:
First of all, when talking about type arithmetic I suggest to refer to the latest presentation for the summit rather than from the Tensor Typing Meetings (https://docs.google.com/presentation/d/1NZzZfbUCXi78LDDe8C3hAaK9on8ZiX3Z2k4B...). Here I intentionally refer to the +,-,*,// syntax rather than the Add[] one. Moreover, I also write 1 + A rather than Literal[1] + A, assuming that we could write a PEP for enabling this behavior if needed, and as Matthew said, it seems feasible based on what was said in the summit.
This PEP sounds like it is of some interest to many of us. Would it make sense to start a repository with an initial document that we can iterate on, or how do you typically develop these? StƩfan
Hi StƩfan, To what potential PEP are you referring exactly? Type arithmetic or automatically interpret `1` as `Literal[1]` wherever it makes sense? Indeed to have a common repository with an initial document would be a good way to go. So far we did not start anything because the PEP on Variadics was still waiting for appoval. However, we also wanted to make sure that Numpy folks believe that this approach towards type arithmetic makes sense for their use cases. Maybe we could sync on this on the next Numpy Development meeting? Best, Alfonso. Quoting Stefan van der Walt <stefanv@berkeley.edu>:
Hi Alfonso, Matthew, Ā On Tue, Jun 15, 2021, at 06:59, ALFONSO LUIS CASTAĆO MARIN wrote:
Ā First of all, when talking about type arithmetic I suggest to refer to the latest presentation for the summit rather than from the Tensor Typing Meetings (https://docs.google.com/presentation/d/1NZzZfbUCXi78LDDe8C3hAaK9on8ZiX3Z2k4B...). Here I intentionally refer to the +,-,*,// syntax rather than the Add[] one. Moreover, I also write 1 + A rather than Literal[1] + A, assuming that we could write a PEP for enabling this behavior if needed, and as Matthew said, it seems feasible based on what was said in the summit.
Ā This PEP sounds like it is of some interest to many of us.Ā Would it make sense to start a repository with an initial document that we can iterate on, or how do you typically develop these? Ā StĆ©fan
On Tue, Jun 15, 2021, at 11:12, ALFONSO LUIS CASTAĆO MARIN wrote:
To what potential PEP are you referring exactly? Type arithmetic or automatically interpret `1` as `Literal[1]` wherever it makes sense?
I guess both, but particularly the first (using standard operators in type definitions). The first seems pretty unwieldy without the second, though.
Indeed to have a common repository with an initial document would be a good way to go. So far we did not start anything because the PEP on Variadics was still waiting for appoval. However, we also wanted to make sure that Numpy folks believe that this approach towards type arithmetic makes sense for their use cases. Maybe we could sync on this on the next Numpy Development meeting?
We have a "triage" meeting this coming Wednesday at 11am Pacific, which would be a good venue for discussing these details. Best regards, StƩfan
participants (5)
-
ALFONSO LUIS CASTAĆO MARIN -
Juan Nunez-Iglesias -
Matthew Rahtz -
Sebastian Berg -
Stefan van der Walt