Feedback before submission of PEP 661: Sentinel Values
Hello, I've been working on this PEP [1] on and off for over a year. I think it is now nearly ready for submission to the steering council. If anyone would like to review the proposal and provide feedback, that would be most welcome! The current reference implementation, and latest version of the PEP, are in this GitHub repo [2]. References to previous discussion are found in the PEP. - Tal Einat [1] https://peps.python.org/pep-0661/ [2] https://github.com/taleinat/python-stdlib-sentinels
On Sat, 10 Sept 2022 at 22:21, Tal Einat <taleinat@gmail.com> wrote:
Hello,
I've been working on this PEP [1] on and off for over a year. I think it is now nearly ready for submission to the steering council.
From the expectations on sentinels:
The sentinel objects should behave as expected by a sentinel object: When compared using the is operator, it should always be considered identical to itself but never to any other object.
That would be intrinsically true of ANY object. Did you mean the "==" operator? (You mention further down that equality should behave that way, but that identity checks are recommended.) Otherwise, looks decent. ChrisA
On Sat, Sep 10, 2022 at 3:35 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 10 Sept 2022 at 22:21, Tal Einat <taleinat@gmail.com> wrote:
Hello,
I've been working on this PEP [1] on and off for over a year. I think it is now nearly ready for submission to the steering council.
From the expectations on sentinels:
The sentinel objects should behave as expected by a sentinel object: When compared using the is operator, it should always be considered identical to itself but never to any other object.
That would be intrinsically true of ANY object. Did you mean the "==" operator? (You mention further down that equality should behave that way, but that identity checks are recommended.)
Yes, I should likely clarify that, thanks for bringing this up! The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true. This allows surviving copying, pickling + unpickling, etc. On the other hand, sentinels with the same name defined in different modules will be different. Other existing implementations of sentinels differ in this respect, in ways that can be surprising IMO.
Otherwise, looks decent.
Thanks for this input too! - Tal
On Sat, Sep 10, 2022 at 6:58 AM Tal Einat <taleinat@gmail.com> wrote:
The sentinel objects should behave as expected by a sentinel object: When compared using the is operator, it should always be considered identical to itself but never to any other object.
The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true.
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace. I wonder if there's a way to make this a bit more clear in the PEP text -- but don't have any suggestions for edits at the moment :-( It would be nice if they could be true singletons, but that would require a python-global registry of some sort, which is, well, impossible. I would like to see a handful of common sentinels defined. Is there a plan to put a few that are useful for the stdlib in the `sentinel` module? Maybe that doesn't belong in the PEP, but I like the idea of a) every similar use in the stdlib uses the same one and b) having a standard set for third party code to use -- a bit like the built-in Exceptions. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I haven't read the PEP yet, so this should not be read as either support of or opposition to the design, just commenting on one aspect. On Sat, Sep 10, 2022 at 01:42:38PM -0700, Christopher Barker wrote:
The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true.
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace. [...] It would be nice if they could be true singletons, but that would require a python-global registry of some sort, which is, well, impossible.
Easy peasey. Add a second, optional, parameter to the Sentinel callable (class?) to specify the namespace, defaulting to "the current module", then register the object to `builtins`. Of course, interpreter-global state should be discouraged, except for the interpreter itself :-) so we probably shouldn't do that. But a namespace parameter will have another advantage: it will allow packages to refactor their initialisation code to a sub-module: ``` # package/__init__.py from package.setup import * # package/setup.py import package import sentinel MySentinel = sentinel.Sentinel("name", namespace=package) ``` Yes, that's a circular import, but I think it is a safe one.
I would like to see a handful of common sentinels defined. Is there a plan to put a few that are useful for the stdlib in the `sentinel` module?
We already have None. What other "common sentinels" do you expect to be useful across the stdlib?
Maybe that doesn't belong in the PEP, but I like the idea of
a) every similar use in the stdlib uses the same one
Do you have examples of such stdlib "similar use" of sentinels? Apart from None of course. -- Steve
On Sun, Sep 11, 2022 at 4:55 AM Steven D'Aprano <steve@pearwood.info> wrote:
I haven't read the PEP yet, so this should not be read as either support of or opposition to the design, just commenting on one aspect.
Given what you've written, I recommend reading it. It's relatively short and directly addresses what you've brought up here.
On Sat, Sep 10, 2022 at 01:42:38PM -0700, Christopher Barker wrote:
The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true.
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace. [...] It would be nice if they could be true singletons, but that would require a python-global registry of some sort, which is, well, impossible.
Easy peasey. Add a second, optional, parameter to the Sentinel callable (class?) to specify the namespace, defaulting to "the current module", then register the object to `builtins`.
This already exists, an optional "module_name" argument, which defaults to the current module (via frame inspection). Using the same name and module name will result in the same sentinel object.
Do you have examples of such stdlib "similar use" of sentinels? Apart from None of course.
The PEP includes a link to a list of sentinel values in the stdlib [1]. - Tal [1] https://mail.python.org/archives/list/python-dev@python.org/message/JBYXQH3N...
Christopher Barker writes:
The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true.
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace.
I don't see why this is a problem. Sentinels are "out of band" signals, they're invalid data *for the problem at hand*. That doesn't mean they're *always* invalid data. For example, Unicode surrogate characters are invalid in conforming streams of characters like Python str, but Python uses them internally anyway as "sentinels" meaning "byte value #XX encountered in input, but is invalid there". (You might not agree that these are sentinels, but I'm hard put to see why my interpretation is incorrect.) And, of course, Unicode itself uses them in concrete representations of characters, specifically UTF-16. So sentinel usage is fundamentally private to an application as far as I can see, and the sentinel value is fundamentally arbitrary, as long as it's not valid data for the application.
I would like to see a handful of common sentinels defined. Is there a plan to put a few that are useful for the stdlib in the `sentinel` module?
I don't think this works. You end up with the problem that None has as a sentinel: mostly it's great, but occasionally application's interpretation gets in the way of the sentinel semantics because in the end Python considers it a valid datum. For modules to be composable, they need to agree on the *exact* semantics of the sentinel. If a convention based on one of the true singletons (None, Ellipsis, True, False) doesn't work, you probably want a private sentinel anyway. IMO YMMV
On Sun, Sep 11, 2022 at 5:30 AM Stephen J. Turnbull <stephenjturnbull@gmail.com> wrote:
Christopher Barker writes:
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace.
I don't see why this is a problem. Sentinels are "out of band" signals, they're invalid data *for the problem at hand*. That doesn't mean they're *always* invalid data. For example, Unicode surrogate characters are invalid in conforming streams of characters like Python str, but Python uses them internally anyway as "sentinels" meaning "byte value #XX encountered in input, but is invalid there". (You might not agree that these are sentinels, but I'm hard put to see why my interpretation is incorrect.) And, of course, Unicode itself uses them in concrete representations of characters, specifically UTF-16.
So sentinel usage is fundamentally private to an application as far as I can see, and the sentinel value is fundamentally arbitrary, as long as it's not valid data for the application.
Indeed! The PEP mentions -1 as return by str.find() as another example.
I would like to see a handful of common sentinels defined. Is there a plan to put a few that are useful for the stdlib in the `sentinel` module?
I don't think this works. You end up with the problem that None has as a sentinel: mostly it's great, but occasionally application's interpretation gets in the way of the sentinel semantics because in the end Python considers it a valid datum. For modules to be composable, they need to agree on the *exact* semantics of the sentinel. If a convention based on one of the true singletons (None, Ellipsis, True, False) doesn't work, you probably want a private sentinel anyway.
This mirrors much of my thinking. Adding an additional sentinel value for general use, similar to None and Ellipses, would be entirely separate from this proposal. The intention here is to allow every function, class or module which needs a sentinel value to define its own unique sentinel(s). I think this is better in most cases, since it makes things clear and well separated. (I arrived at this conclusion by going through many examples of existing code using sentinel values, both in the stdlib and elsewhere.) - Tal
On Sat, Sep 10, 2022 at 11:42 PM Christopher Barker <pythonchb@gmail.com> wrote:
On Sat, Sep 10, 2022 at 6:58 AM Tal Einat <taleinat@gmail.com> wrote:
The current design is that sentinels with the same name from the same module will always be identical. So for example `Sentinel("name") is Sentinel("name")` will be true.
Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only in a particular module namespace.
You're right, these aren't singletons. I've been careful not to use the word "Singleton" in the PEP and relevant discussion.
I would like to see a handful of common sentinels defined. Is there a plan to put a few that are useful for the stdlib in the `sentinel` module?
No. The main use cases driving this design are where a *distinct* value is needed, that would not be used anywhere else for any other use. I'll Stephen J. Turnbull put this nicely in a later reply: "If a convention based on one of the true singletons (None, Ellipsis, True, False) doesn't work, you probably want a private sentinel anyway."
b) having a standard set for third party code to use -- a bit like the built-in Exceptions.
As far as I know, the main reason for the existence of Python's built-in exceptions is that they are used by Python's internals and stdlib, not as a standardized set of exceptions for users. If it was the other way around, we'd probably have a much more extensive tree of built-in exceptions. Other languages (Java comes to mind) have gone further in this direction. To my understanding, that has been avoided intentionally in Python. - Tal
participants (5)
-
Chris Angelico -
Christopher Barker -
Stephen J. Turnbull -
Steven D'Aprano -
Tal Einat