Introducing `np.types`
Hi all,
I was wondering if we should introduce a new `np.types` namespace. The main reason is that we have the DType classes, that most users don't need to worry about. These mirror the scalar classes, but getting them is weird currently.
I never wanted to put these in the top-level (because I feel they probably won't be used much day to day). That would be thing like:
* np.types.IntDType, np.types.Int64DType (or maybe without dtype) * np.types.NumberDType (an abstract DType) * np.types.InexactDType * ... * np.types.DTypeMeta (the metaclass used, just to have it somewhere)
Maybe there are some more types that we could use a public entrypoint for (e.g. the type used by array-function dispatched functions or `np.ufunc` could in principle move also).
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
(I will note that the DType classes do get printed sometimes in error messages.)
Cheers,
Sebastian
On Fri, Feb 10, 2023 at 3:31 AM Sebastian Berg sebastian@sipsolutions.net wrote:
Hi all,
I was wondering if we should introduce a new `np.types` namespace. The main reason is that we have the DType classes, that most users don't need to worry about. These mirror the scalar classes, but getting them is weird currently.
I never wanted to put these in the top-level (because I feel they probably won't be used much day to day). That would be thing like:
- np.types.IntDType, np.types.Int64DType (or maybe without dtype)
- np.types.NumberDType (an abstract DType)
- np.types.InexactDType
- ...
- np.types.DTypeMeta (the metaclass used, just to have it somewhere)
Maybe there are some more types that we could use a public entrypoint for (e.g. the type used by array-function dispatched functions or `np.ufunc` could in principle move also).
Small bikeshed: the name np.types indicates to me that it has something to do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDType
I know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
On Fri, Feb 10, 2023 at 5:35 PM Nathan nathan.goldbaum@gmail.com wrote:
On Fri, Feb 10, 2023 at 3:31 AM Sebastian Berg sebastian@sipsolutions.net wrote:
Hi all,
I was wondering if we should introduce a new `np.types` namespace. The main reason is that we have the DType classes, that most users don't need to worry about. These mirror the scalar classes, but getting them is weird currently.
I never wanted to put these in the top-level (because I feel they probably won't be used much day to day). That would be thing like:
- np.types.IntDType, np.types.Int64DType (or maybe without dtype)
- np.types.NumberDType (an abstract DType)
- np.types.InexactDType
- ...
- np.types.DTypeMeta (the metaclass used, just to have it somewhere)
Maybe there are some more types that we could use a public entrypoint for (e.g. the type used by array-function dispatched functions or `np.ufunc` could in principle move also).
Small bikeshed: the name np.types indicates to me that it has something to do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
I had the same concern. It would be good to have a full list of things to put in this and the scope of it. There's other things that could fit, like custom warning and exception classes. We just added `numpy.exceptions` for the next release, but that could still be moved. Having separate submodules for type classes, for exceptions, for array-function override related stuff, etc. is not great. Enums could fit as well, they're now polluting the main namespace. It seems like we need one namespace with all that kind of stuff that the average end user won't need but that becomes useful/important when you're doing something quite custom or are writing a library on top of numpy.
Cheers, Ralf
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDTypeI know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ralf.gommers@googlemail.com
On Sat, 2023-02-11 at 11:24 +0000, Ralf Gommers wrote:
On Fri, Feb 10, 2023 at 5:35 PM Nathan nathan.goldbaum@gmail.com wrote:
<snip>
Small bikeshed: the name np.types indicates to me that it has something to do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
I had the same concern. It would be good to have a full list of things to put in this and the scope of it. There's other things that could fit, like custom warning and exception classes. We just added `numpy.exceptions` for the next release, but that could still be moved. Having separate submodules for type classes, for exceptions, for array-function override related stuff, etc. is not great. Enums could fit as well, they're now polluting the main namespace. It seems like we need one namespace with all that kind of stuff that the average end user won't need but that becomes useful/important when you're doing something quite custom or are writing a library on top of numpy.
Well, right now I care mainly about dtype related types. Other possible ones might be: * np.ufunc (just because I doubt it is used much at all) * the array-function dispatcher (private right now) * DTypeMeta (dtype related, though)
But, not sure we have many more. I don't think I mind `exceptions` to be a separate namespace it seems nice and clear? For the rest, either a dtype specific solution or not is fine by me.
I suggested types because Python has `types` as a catch-all for builtin types (mainly function, generator, ...) that are not normally used directly and it seemed relatively clear and concise. (In Python I don't think it has anything to do with typing directly.)
- Sebastian
Cheers, Ralf
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDType
I know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ralf.gommers@googlemail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On Mon, Feb 13, 2023 at 4:34 AM Sebastian Berg sebastian@sipsolutions.net wrote:
On Sat, 2023-02-11 at 11:24 +0000, Ralf Gommers wrote:
On Fri, Feb 10, 2023 at 5:35 PM Nathan nathan.goldbaum@gmail.com wrote:
<snip>
Small bikeshed: the name np.types indicates to me that it has something to do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
I had the same concern. It would be good to have a full list of things to put in this and the scope of it. There's other things that could fit, like custom warning and exception classes. We just added `numpy.exceptions` for the next release, but that could still be moved. Having separate submodules for type classes, for exceptions, for array-function override related stuff, etc. is not great. Enums could fit as well, they're now polluting the main namespace. It seems like we need one namespace with all that kind of stuff that the average end user won't need but that becomes useful/important when you're doing something quite custom or are writing a library on top of numpy.
Well, right now I care mainly about dtype related types. Other possible ones might be:
- np.ufunc (just because I doubt it is used much at all)
- the array-function dispatcher (private right now)
- DTypeMeta (dtype related, though)
But, not sure we have many more. I don't think I mind `exceptions` to be a separate namespace it seems nice and clear? For the rest, either a dtype specific solution or not is fine by me.
I suggested types because Python has `types` as a catch-all for builtin types (mainly function, generator, ...) that are not normally used directly and it seemed relatively clear and concise. (In Python I don't think it has anything to do with typing directly.)
Ah, good analogy! I didn’t think of that.
I think moving to the future where dtype classes have sensible names and reprs, and can be accesses straightforwardly at runtime without doing type(dtype_instance) is a real improvement and thank you for pushing on this. I don’t think how it’s spelled is as important.
- Sebastian
Cheers, Ralf
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDTypeI know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ralf.gommers@googlemail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
On Mon, Feb 13, 2023 at 11:33 AM Sebastian Berg sebastian@sipsolutions.net wrote:
On Sat, 2023-02-11 at 11:24 +0000, Ralf Gommers wrote:
On Fri, Feb 10, 2023 at 5:35 PM Nathan nathan.goldbaum@gmail.com wrote:
<snip>
Small bikeshed: the name np.types indicates to me that it has something to do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
I had the same concern. It would be good to have a full list of things to put in this and the scope of it. There's other things that could fit, like custom warning and exception classes. We just added `numpy.exceptions` for the next release, but that could still be moved. Having separate submodules for type classes, for exceptions, for array-function override related stuff, etc. is not great. Enums could fit as well, they're now polluting the main namespace. It seems like we need one namespace with all that kind of stuff that the average end user won't need but that becomes useful/important when you're doing something quite custom or are writing a library on top of numpy.
Well, right now I care mainly about dtype related types. Other possible ones might be:
- np.ufunc (just because I doubt it is used much at all)
- the array-function dispatcher (private right now)
- DTypeMeta (dtype related, though)
Those all seem to fit indeed.
But, not sure we have many more. I don't think I mind `exceptions` to be a separate namespace it seems nice and clear? For the rest, either a dtype specific solution or not is fine by me.
I suggested types because Python has `types` as a catch-all for builtin types (mainly function, generator, ...) that are not normally used directly and it seemed relatively clear and concise. (In Python I don't think it has anything to do with typing directly.)
Okay, as long as we keep in mind that it should contain all these not-for-main-namespace functions/classes, it seems fine with me. We can live with two namespaces (`types` and `exceptions`), but more would get a bit too much. We were planning to use `np.lib.*` for more detailed user-facing functions that didn't belong in the main namespace, so let's make sure that we don't end up with >2 of these kinds of namespaces right below the top level one.
Cheers, Ralf
- Sebastian
Cheers, Ralf
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDTypeI know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ralf.gommers@googlemail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On Tue, Feb 14, 2023, at 12:27, Ralf Gommers wrote:
Okay, as long as we keep in mind that it should contain all these not-for-main-namespace functions/classes, it seems fine with me. We can live with two namespaces (`types` and `exceptions`), but more would get a bit too much. We were planning to use `np.lib.*` for more detailed user-facing functions that didn't belong in the main namespace, so let's make sure that we don't end up with >2 of these kinds of namespaces right below the top level one.
If it's mainly for internal use, we can also use sub-namespaces under lib right now, and not add those to the main namespace with the next release? I.e., we can create `lib.types`, `lib.exceptions` and whatever we want, and then import lib as _np or similar internally.
Stéfan
On Tue, 2023-02-14 at 13:07 -0800, Stefan van der Walt wrote:
On Tue, Feb 14, 2023, at 12:27, Ralf Gommers wrote:
Okay, as long as we keep in mind that it should contain all these not-for-main-namespace functions/classes, it seems fine with me. We can live with two namespaces (`types` and `exceptions`), but more would get a bit too much. We were planning to use `np.lib.*` for more detailed user-facing functions that didn't belong in the main namespace, so let's make sure that we don't end up with >2 of these kinds of namespaces right below the top level one.
If it's mainly for internal use, we can also use sub-namespaces under lib right now, and not add those to the main namespace with the next release? I.e., we can create `lib.types`, `lib.exceptions` and whatever we want, and then import lib as _np or similar internally.
Do you (or anyone else) have a preference? Using `np.lib.<submodule>` for future things especially if they don't feel like core functionality seems good. (I am not a big fan of having a second large namespace, but have little preference for where we would would put new submodules, i.e. in `np.lib`.<submodule> or `np.<submodule>` directly.)
For these particular ones, I still slightly prefer `np.types` and `np.exceptions`, but happy either way. One disadvantage of the main namespace is that it is so large, it is hard to find submodules :).
- Sebastian
Stéfan _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On Tue, Feb 14, 2023 at 9:08 PM Stefan van der Walt stefanv@berkeley.edu wrote:
On Tue, Feb 14, 2023, at 12:27, Ralf Gommers wrote:
Okay, as long as we keep in mind that it should contain all these not-for-main-namespace functions/classes, it seems fine with me. We can live with two namespaces (`types` and `exceptions`), but more would get a bit too much. We were planning to use `np.lib.*` for more detailed user-facing functions that didn't belong in the main namespace, so let's make sure that we don't end up with >2 of these kinds of namespaces right below the top level one.
If it's mainly for internal use, we can also use sub-namespaces under lib right now, and not add those to the main namespace with the next release? I.e., we can create `lib.types`, `lib.exceptions` and whatever we want, and then import lib as _np or similar internally.
It's not for internal use, the point of these proposals is to expose public namespaces that contain objects needed by third-party packages. So the current proposal seems better than throwing it into the `np.lib.*` mix.
Cheers, Ralf
On Fri, 2023-02-10 at 10:34 -0700, Nathan wrote:
On Fri, Feb 10, 2023 at 3:31 AM Sebastian Berg < sebastian@sipsolutions.net> wrote:
Hi all,
I was wondering if we should introduce a new `np.types` namespace. The main reason is that we have the DType classes, that most users don't need to worry about. These mirror the scalar classes, but getting them is weird currently.
I never wanted to put these in the top-level (because I feel they probably won't be used much day to day). That would be thing like:
- np.types.IntDType, np.types.Int64DType (or maybe without dtype)
- np.types.NumberDType (an abstract DType)
- np.types.InexactDType
- ...
- np.types.DTypeMeta (the metaclass used, just to have it
somewhere)
Maybe there are some more types that we could use a public entrypoint for (e.g. the type used by array-function dispatched functions or `np.ufunc` could in principle move also).
Small bikeshed: the name np.types indicates to me that it has something to
We tried discussing it in the community meeting today and it led to bike-shedding. So lets see if a poll can give an idea for a preferred color:
https://discuss.scientific-python.org/t/where-to-put-dtype-classes/655
Happy to add more options (or just get posts there). I don't care too much overall about the where, I just feel it is well time to put it somewhere.
- Sebastian
do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDType
I know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
I'm not sure I understand the objection to having "too many submodules", but if it's a concern, the exceptions could be added here instead of a separate np.exceptions, since exceptions are themselves just types.
Aaron Meurer
On Wed, Feb 15, 2023 at 2:33 PM Sebastian Berg sebastian@sipsolutions.net wrote:
On Fri, 2023-02-10 at 10:34 -0700, Nathan wrote:
On Fri, Feb 10, 2023 at 3:31 AM Sebastian Berg < sebastian@sipsolutions.net> wrote:
Hi all,
I was wondering if we should introduce a new `np.types` namespace. The main reason is that we have the DType classes, that most users don't need to worry about. These mirror the scalar classes, but getting them is weird currently.
I never wanted to put these in the top-level (because I feel they probably won't be used much day to day). That would be thing like:
- np.types.IntDType, np.types.Int64DType (or maybe without dtype)
- np.types.NumberDType (an abstract DType)
- np.types.InexactDType
- ...
- np.types.DTypeMeta (the metaclass used, just to have it
somewhere)
Maybe there are some more types that we could use a public entrypoint for (e.g. the type used by array-function dispatched functions or `np.ufunc` could in principle move also).
Small bikeshed: the name np.types indicates to me that it has something to
We tried discussing it in the community meeting today and it led to bike-shedding. So lets see if a poll can give an idea for a preferred color:
https://discuss.scientific-python.org/t/where-to-put-dtype-classes/655
Happy to add more options (or just get posts there). I don't care too much overall about the where, I just feel it is well time to put it somewhere.
- Sebastian
do with static typing. If this namespace only includes dtype classes, then np.dtype_classes is a more natural name. If it includes things like `np.ufunc` then that's not as clear, and I don't have a better idea offhand than np.types.
What do you think? I don't really have a good idea for an alternative but at some point not making these nicely public is not great...
Related to your proposal but not orthogonal to it, I still think it would still be nice to be able to do things like:
>>> np.dtype[numbers.Number] np.types.NumberDTypeI know that currently __class_getitem__ is used by the typing support, but I think the typing will still work if you also got back a usable dtype instance at runtime instead of a GenericAlias, which has a confusing repr and is not useful at runtime.
(I will note that the DType classes do get printed sometimes in error messages.)
See also https://github.com/numpy/numpy/issues/22920.
Cheers,
Sebastian
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: nathan12343@gmail.com
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: asmeurer@gmail.com
participants (5)
-
Aaron Meurer -
Nathan -
Ralf Gommers -
Sebastian Berg -
Stefan van der Walt