Following a recent change, we now have in traceback.py: _sentinel = object() def print_exception(exc, /, value=_sentinel, tb=_sentinel, limit=None, file=None, chain=True): So now: >>> import traceback >>> help(traceback.print_exception) Help on function print_exception in module traceback: print_exception(exc, /, value=<object object at 0x000002825DF09650>, tb=<object object at 0x000002825DF09650>, limit=None, file=None, chain=True) Is there a convention on how such default sentinel values should appear in docs? https://bugs.python.org/issue43024
On Thu, 13 May 2021 10:15:03 +0100 Irit Katriel via Python-Dev <python-dev@python.org> wrote:
Following a recent change, we now have in traceback.py:
_sentinel = object() def print_exception(exc, /, value=_sentinel, tb=_sentinel, limit=None, file=None, chain=True):
So now:
>>> import traceback >>> help(traceback.print_exception) Help on function print_exception in module traceback:
print_exception(exc, /, value=<object object at 0x000002825DF09650>, tb=<object object at 0x000002825DF09650>, limit=None, file=None, chain=True)
Is there a convention on how such default sentinel values should appear in docs?
If this were a positional-only argument, you could use square brackets, e.g.: print_exception(exc[, value[, ...]]) Other than that, I can't think of any existing convention. I agree that <optional> is a reasonable spelling. Regards Antoine.
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>. The arg is only there for backwards compatibility now.
Le 13/05/2021 à 11:40, Irit Katriel a écrit :
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org <mailto:antoine@python.org>> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>.
"<derived>" makes think of a derived class, and leaves me confused. "<implicit>" is a bit better, but doesn't clearly say what the default value is, either. So in all cases I have to read the docstring in addition to the function signature. Regards Antoine.
On 13. 05. 21 11:45, Antoine Pitrou wrote:
Le 13/05/2021 à 11:40, Irit Katriel a écrit :
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org <mailto:antoine@python.org>> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>.
"<derived>" makes think of a derived class, and leaves me confused. "<implicit>" is a bit better, but doesn't clearly say what the default value is, either. So in all cases I have to read the docstring in addition to the function signature.
Is <default> the term you're looking for?
On 5/13/2021 7:48 AM, Petr Viktorin wrote:
On 13. 05. 21 11:45, Antoine Pitrou wrote:
Le 13/05/2021 à 11:40, Irit Katriel a écrit :
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org <mailto:antoine@python.org>> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>.
"<derived>" makes think of a derived class, and leaves me confused. "<implicit>" is a bit better, but doesn't clearly say what the default value is, either. So in all cases I have to read the docstring in addition to the function signature.
Is <default> the term you're looking for?
In the dataclasses docs https://docs.python.org/3/library/dataclasses.html I document the module-level sentinel MISSING, then I document the function as: dataclasses.field(*, default=MISSING, default_factory=MISSING, repr=True, hash=None, init=True, compare=True, metadata=None) And I explain what MISSING means. The help looks like: field(*, default=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, default_factory=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, init=True, repr=True, hash=None, compare=True, metadata=None) None of this is particularly awesome, but no one has complained about it yet. I think it's important to state an actual value for the default value, instead of just using something like "<default>". Unless you know the actual default value, you can't write a wrapper. Say I wanted to write something that calls dataclasses.field, but doesn't allow the user to specify repr, hash, init, compare, and metadata. I could write it as: def myfield_func(*, default=MISSING, default_factory=MISSING): ... If the real default values for "default" and "default_factory" weren't documented, there wouldn't be any easy way to write this. I do think a python-wide standard for this would be helpful, but I don't see how to change existing code given backward compatibility constraints. Eric
On Thu, May 13, 2021 at 4:31 PM Eric V. Smith <eric@trueblade.com> wrote:
I do think a python-wide standard for this would be helpful, but I don't see how to change existing code given backward compatibility constraints.
While we're on the subject, these sentinels also don't compare properly using `is` after pickling and unpickling. I think it's worth considering making the sentinels in the stdlib all have good reprs and support pickling+unpickling. What would be the potential backwards-compatibility issues with changing the implementation of these existing sentinel values? - Tal
On 5/13/2021 10:02 AM, Tal Einat wrote:
I do think a python-wide standard for this would be helpful, but I don't see how to change existing code given backward compatibility constraints. While we're on the subject, these sentinels also don't compare
On Thu, May 13, 2021 at 4:31 PM Eric V. Smith <eric@trueblade.com> wrote: properly using `is` after pickling and unpickling.
I think it's worth considering making the sentinels in the stdlib all have good reprs and support pickling+unpickling.
What would be the potential backwards-compatibility issues with changing the implementation of these existing sentinel values?
I don't think there would be a problem changing the implementation. I was commenting on changing the name of the sentinel objects so that we could document the functions with the sentinel's real name. We couldn't change them to all be some appropriate module-level value named "MISSING", for example. Eric
On 5/13/21 2:15 AM, Irit Katriel via Python-Dev wrote:
>>> help(traceback.print_exception) Help on function print_exception in module traceback:
print_exception(exc, /, value=<object object at 0x000002825DF09650>, tb=<object object at 0x000002825DF09650>, limit=None, file=None, chain=True)
On 5/13/21 5:37 AM, Eric V. Smith wrote:
The help looks like:
field(*, default=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, default_factory=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, init=True, repr=True, hash=None, compare=True, metadata=None)
None of this is particularly awesome, but no one has complained about it yet.
Consider me complaining. ;-) Looks to me like the default repr for the sentinels is making those helps much less helpful by showing totally irrelevant information and cluttering up the screen making it harder to see the actually useful bits. An actual Sentinel class would be helpful: >>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ... >>> MISSING = Sentinel('MISSING') >>> MISSING MISSING >>> implicit = Sentinel('<implicit>') >>> implicit <implicit> Naturally, since sentinels are symbolic names, I think it should go into the enum module. ;-) Although I will concede that we could just put those five lines into the modules that need it. -- ~Ethan~
On Thu, May 13, 2021 at 7:44 PM Ethan Furman <ethan@stoneleaf.us> wrote:
Consider me complaining. ;-)
+1
An actual Sentinel class would be helpful:
>>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ...
>>> MISSING = Sentinel('MISSING') >>> MISSING MISSING
>>> implicit = Sentinel('<implicit>') >>> implicit <implicit>
Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr: class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated') class MISSING(Sentinel): pass - Tal
On 5/13/2021 1:39 PM, Tal Einat wrote:
On Thu, May 13, 2021 at 7:44 PM Ethan Furman <ethan@stoneleaf.us> wrote:
Consider me complaining. ;-) +1
An actual Sentinel class would be helpful:
>>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ...
>>> MISSING = Sentinel('MISSING') >>> MISSING MISSING
>>> implicit = Sentinel('<implicit>') >>> implicit <implicit> Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr:
class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated')
class MISSING(Sentinel): pass
MISSING <class '__main__.MISSING'>
I think a repr of just "MISSING", or maybe "dataclasses.MISSING" would be better. Eric
On 5/13/21 10:46 AM, Eric V. Smith wrote:
MISSING <class '__main__.MISSING'>
I think a repr of just "MISSING", or maybe "dataclasses.MISSING" would be better.
I literally just went down this road--for a while there was a special sentinel value for the eval_str parameter to inspect.get_annotations(). The repr I went with was "<id>", e.g "<MISSING>". It depends on how seriously you take the idea that eval(repr(x)) == x. Certainly most objects don't actually support that, e.g., uh, object(), a type which I understand is available in most Python implementations. Cheers, //arry/
On Thu, May 13, 2021 at 8:46 PM Eric V. Smith <eric@trueblade.com> wrote:
On 5/13/2021 1:39 PM, Tal Einat wrote:
Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr:
class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated')
class MISSING(Sentinel): pass
MISSING <class '__main__.MISSING'>
I think a repr of just "MISSING", or maybe "dataclasses.MISSING" would be better.
The repr uses whatever module the class is defined in, so we'd get:
from dataclasses import MISSING MISSING <class 'dataclasses.MISSING'>
We could override that to something even cleaner with a meta-class. For example: class Sentinel(type): @classmethod def __prepare__(cls, name, bases, **kwds): d = super().__prepare__(name, bases, **kwds) def __new__(cls_, *args, **kwargs): raise TypeError(f'{cls_!r} is a sentinel and cannot be instantiated') d.update(__new__=__new__) return d def __repr__(cls): return f'{cls.__module__}.{cls.__qualname__}' Which results in:
from dataclasses import MISSING MISSING dataclasses.MISSING type(MISSING) <class 'sentinels.Sentinel'> MISSING() Traceback (most recent call last): ... TypeError: dataclasses.MISSING is a sentinel and cannot be instantiated
- Tal - Tal
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None. I can't remember the context, but the consensus seemed to be that it was easy to create a custom sentinel object, and it was not worth adding more "official" ones to the language. But this conversation reminded me about that, and while I do agree that we don't need more that are elevated to the status of None, maybe it would be good to have a couple (or only MISSING) in the standard library somewhere "central" for everyone to use. "central" rather than in, say, dataclasses. I'm not sure where that should be, the operator module, maybe?? Ayway, if someone were to put one of the nifty implementations being discussed here in the stdlib --I'd use it :-) -CHB On Thu, May 13, 2021 at 1:14 PM Tal Einat <taleinat@gmail.com> wrote:
On Thu, May 13, 2021 at 8:46 PM Eric V. Smith <eric@trueblade.com> wrote:
On 5/13/2021 1:39 PM, Tal Einat wrote:
Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr:
class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated')
class MISSING(Sentinel): pass
MISSING <class '__main__.MISSING'>
I think a repr of just "MISSING", or maybe "dataclasses.MISSING" would be better.
The repr uses whatever module the class is defined in, so we'd get:
from dataclasses import MISSING MISSING <class 'dataclasses.MISSING'>
We could override that to something even cleaner with a meta-class. For example:
class Sentinel(type): @classmethod def __prepare__(cls, name, bases, **kwds): d = super().__prepare__(name, bases, **kwds) def __new__(cls_, *args, **kwargs): raise TypeError(f'{cls_!r} is a sentinel and cannot be instantiated') d.update(__new__=__new__) return d
def __repr__(cls): return f'{cls.__module__}.{cls.__qualname__}'
Which results in:
from dataclasses import MISSING MISSING dataclasses.MISSING type(MISSING) <class 'sentinels.Sentinel'> MISSING() Traceback (most recent call last): ... TypeError: dataclasses.MISSING is a sentinel and cannot be instantiated
- Tal
- Tal _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ECYFQWPB... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Thu, May 13, 2021 at 10:31 PM Christopher Barker <pythonchb@gmail.com> wrote:
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None.
As I remember, the year-ago conversation was basically wanting more ways of saying "argument not specified" in function signatures. The thought was that None is used pretty often to mean something somewhat different. The rough consensus seemed to be that `my_sentinel = object()` was a low burden per project. But in what I recall, there was no talk there of custom behavior like a nicer repr(). My own feeling is that ONLY a repr() isn't quite enough to motivate an addition to stdlib. But if there were a few other useful behaviors, maybe it would be (e.g. maybe default or specifiable inequality operations). However, I wouldn't really want another two or three or four singletons, but rather a slightly more templated factory than just `object()`. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None.
Not quite true, we also have Ellipsis, which already has a nice repr that both reads easily and still follows the convention of eval(repr(x)) == x. It also is already safe from instantiation, survives pickle round-trip and is multi-thread safe. So long as you are not dealing with scientific projects, it seems a quick (if dirty) solution to having a sentinel that is not None. There is also some symmetrical wholeness when considered with the other builtin sentinels: bool(None) is False; bool(Ellipsis) is True.
On 14May2021 0622, micro codery wrote:
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None.
Not quite true, we also have Ellipsis, which already has a nice repr that both reads easily and still follows the convention of eval(repr(x)) == x. It also is already safe from instantiation, survives pickle round-trip and is multi-thread safe. So long as you are not dealing with scientific projects, it seems a quick (if dirty) solution to having a sentinel that is not None.
I don't think using "..." to indicate "some currently unknown or unspecified value" is dirty at all, it seems perfectly consistent with how we use it in English (and indexing in scientific projects, for that matter, where it tends to imply "figure out the rest for me"). All that's really missing is some kind of endorsement (from python-dev, presumably in the docs) that it's okay to use it as a default parameter value. I can't think of any reason you'd need to accept Ellipsis as a *specified* value that wouldn't also apply to any other kind of shared sentinel. Cheers, Steve
On Fri, May 14, 2021 at 12:45 PM Steve Dower <steve.dower@python.org> wrote:
On 14May2021 0622, micro codery wrote:
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None.
Not quite true, we also have Ellipsis, which already has a nice repr that both reads easily and still follows the convention of eval(repr(x)) == x. It also is already safe from instantiation, survives pickle round-trip and is multi-thread safe. So long as you are not dealing with scientific projects, it seems a quick (if dirty) solution to having a sentinel that is not None.
I don't think using "..." to indicate "some currently unknown or unspecified value" is dirty at all, it seems perfectly consistent with how we use it in English (and indexing in scientific projects, for that matter, where it tends to imply "figure out the rest for me").
All that's really missing is some kind of endorsement (from python-dev, presumably in the docs) that it's okay to use it as a default parameter value. I can't think of any reason you'd need to accept Ellipsis as a *specified* value that wouldn't also apply to any other kind of shared sentinel.
I'll try to organize my thoughts a bit here. This is a bit long, welcome to skip to the final sentence for the "tl;dr". Features one may want for a sentinel: 1. Unique from other objects 2. Globally unique, i.e. unique from other such sentinels (no consensus here) 3. Clear repr (the original subject of this thread) - significant since these often appear in function signatures 4. Survives pickling round-trip (I brought this up since I was bitten by this once, but others have mentioned that this is usually irrelevant) 5. Can be given a clear type signature (this was brought up on twitter[1]) - significant since without this nobody can add full type signatures even if they want to The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread. I'd also personally prefer something simple, ideally without a new class or module. There are several simple idioms using existing features that seem like good options, so let's review those: 1. Ellipsis, a.k.a. `...`. This has all of the features outlined above except #2. My main issue with using Ellipsis for this is that it could be surprising and confusing for devs first encountering such use, and could be relatively awkward to figure out. 2. An instance of a one-off class. dataclasses.MISSING is an example of this. It is defined thus: class _MISSING: pass MISSING = _MISSING() Besides failing #4 (surviving pickle round-trips), its repr isn't great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily overcome by implementing __repr__. 3. A one-off class: class MISSING: pass This has all of the above features except #5: having a clear type signature (since its type is type). Using a class as a value this way could be surprising, though. It's repr also isn't great: <class 'dataclasses._MISSING'>. 4. A value of an single-valued enum, for example (from [1]): class _UNSET(enum.Enum): token = enum.auto() This has all of the features above and is simple, just requiring a comment to explain what it is. It's repr is a bit awkward though:
repr(_UNSET.token) '<_UNSET.token: 1>'
All of these are in use by some developers, though not necessarily in the stdlib. None is perfect, though all are probably good enough. Since pickling is likely not relevant in most cases, I'm currently in favor of #2 making sure to implement a nice __repr__. - Tal [1] https://twitter.com/nolar/status/1392962447166877697
If we drop the requirement for pickle round-tripping then I would add a requirement that sentinel is unpicklable, to prevent accidents. Irit On Fri, May 14, 2021 at 8:38 PM Tal Einat <taleinat@gmail.com> wrote:
I'll try to organize my thoughts a bit here. This is a bit long, welcome to skip to the final sentence for the "tl;dr".
Features one may want for a sentinel: 1. Unique from other objects 2. Globally unique, i.e. unique from other such sentinels (no consensus here) 3. Clear repr (the original subject of this thread) - significant since these often appear in function signatures 4. Survives pickling round-trip (I brought this up since I was bitten by this once, but others have mentioned that this is usually irrelevant) 5. Can be given a clear type signature (this was brought up on twitter[1]) - significant since without this nobody can add full type signatures even if they want to
The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread.
I'd also personally prefer something simple, ideally without a new class or module.
There are several simple idioms using existing features that seem like good options, so let's review those:
1. Ellipsis, a.k.a. `...`. This has all of the features outlined above except #2. My main issue with using Ellipsis for this is that it could be surprising and confusing for devs first encountering such use, and could be relatively awkward to figure out.
2. An instance of a one-off class. dataclasses.MISSING is an example of this. It is defined thus:
class _MISSING: pass MISSING = _MISSING()
Besides failing #4 (surviving pickle round-trips), its repr isn't great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily overcome by implementing __repr__.
3. A one-off class:
class MISSING: pass
This has all of the above features except #5: having a clear type signature (since its type is type). Using a class as a value this way could be surprising, though. It's repr also isn't great: <class 'dataclasses._MISSING'>.
4. A value of an single-valued enum, for example (from [1]):
class _UNSET(enum.Enum): token = enum.auto()
This has all of the features above and is simple, just requiring a comment to explain what it is. It's repr is a bit awkward though:
repr(_UNSET.token) '<_UNSET.token: 1>'
All of these are in use by some developers, though not necessarily in the stdlib. None is perfect, though all are probably good enough. Since pickling is likely not relevant in most cases, I'm currently in favor of #2 making sure to implement a nice __repr__.
- Tal
[1] https://twitter.com/nolar/status/1392962447166877697 _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BYLTDD72... Code of Conduct: http://python.org/psf/codeofconduct/
I think it's more future-looking to allow pickle round-tripping. Just add a ._uuid attribute and have object equality follow equality of that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept. On Fri, May 14, 2021, 7:01 PM Irit Katriel via Python-Dev < python-dev@python.org> wrote:
If we drop the requirement for pickle round-tripping then I would add a requirement that sentinel is unpicklable, to prevent accidents.
Irit
On Fri, May 14, 2021 at 8:38 PM Tal Einat <taleinat@gmail.com> wrote:
I'll try to organize my thoughts a bit here. This is a bit long, welcome to skip to the final sentence for the "tl;dr".
Features one may want for a sentinel: 1. Unique from other objects 2. Globally unique, i.e. unique from other such sentinels (no consensus here) 3. Clear repr (the original subject of this thread) - significant since these often appear in function signatures 4. Survives pickling round-trip (I brought this up since I was bitten by this once, but others have mentioned that this is usually irrelevant) 5. Can be given a clear type signature (this was brought up on twitter[1]) - significant since without this nobody can add full type signatures even if they want to
The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread.
I'd also personally prefer something simple, ideally without a new class or module.
There are several simple idioms using existing features that seem like good options, so let's review those:
1. Ellipsis, a.k.a. `...`. This has all of the features outlined above except #2. My main issue with using Ellipsis for this is that it could be surprising and confusing for devs first encountering such use, and could be relatively awkward to figure out.
2. An instance of a one-off class. dataclasses.MISSING is an example of this. It is defined thus:
class _MISSING: pass MISSING = _MISSING()
Besides failing #4 (surviving pickle round-trips), its repr isn't great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily overcome by implementing __repr__.
3. A one-off class:
class MISSING: pass
This has all of the above features except #5: having a clear type signature (since its type is type). Using a class as a value this way could be surprising, though. It's repr also isn't great: <class 'dataclasses._MISSING'>.
4. A value of an single-valued enum, for example (from [1]):
class _UNSET(enum.Enum): token = enum.auto()
This has all of the features above and is simple, just requiring a comment to explain what it is. It's repr is a bit awkward though:
repr(_UNSET.token) '<_UNSET.token: 1>'
All of these are in use by some developers, though not necessarily in the stdlib. None is perfect, though all are probably good enough. Since pickling is likely not relevant in most cases, I'm currently in favor of #2 making sure to implement a nice __repr__.
- Tal
[1] https://twitter.com/nolar/status/1392962447166877697 _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BYLTDD72... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7I7S22KE... Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote:
I think it's more future-looking to allow pickle round-tripping.
I tend to agree.
Just add a ._uuid attribute and have object equality follow equality of that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept.
I think it's worth preserving the idiom of comparing sentinels using `is`, as we do for `None` and other existing sentinel values. It's relatively easy to do, such as by using a single-value Enum or by using a class with a custom __new__. - Tal
On Thu, May 20, 2021 at 6:21 AM Tal Einat <taleinat@gmail.com> wrote:
Just add a ._uuid attribute and have object equality follow equality of
On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote: that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept.
I think it's worth preserving the idiom of comparing sentinels using `is`, as we do for `None` and other existing sentinel values. It's relatively easy to do, such as by using a single-value Enum or by using a class with a custom __new__.
This only works if: a) Unpickling is within a single interpreter session b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation Maybe there's a way to do it, but how would you handle this situation: if some_runtime_condition: my_sentinal = Sentinel(desc="Gosh, I need a sentinel") # ... code ... pickle.dump(thing_using_sentinel, fh) Equality is certainly a lot easier to get than identity here. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Fri, May 21, 2021 at 3:51 AM David Mertz <mertz@gnosis.cx> wrote:
On Thu, May 20, 2021 at 6:21 AM Tal Einat <taleinat@gmail.com> wrote:
On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote:
Just add a ._uuid attribute and have object equality follow equality of that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept.
I think it's worth preserving the idiom of comparing sentinels using `is`, as we do for `None` and other existing sentinel values. It's relatively easy to do, such as by using a single-value Enum or by using a class with a custom __new__.
This only works if:
a) Unpickling is within a single interpreter session b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation
Maybe there's a way to do it, but how would you handle this situation:
if some_runtime_condition: my_sentinal = Sentinel(desc="Gosh, I need a sentinel") # ... code ... pickle.dump(thing_using_sentinel, fh)
Equality is certainly a lot easier to get than identity here.
Probably the easiest way would be to have some kind of unique identifier (a fully-qualified name) that can be pickled, and then any time you attempt to construct a Sentinel with that identifier, it's guaranteed to return the same object. That way, in your example, unpickling it in another session where that runtime condition is false would simply create a brand new sentinel, which would be the same as any others that also get unpickled; and if, subsequently, you reconstruct my_sentinal, it would be the same one as got unpickled. The hardest part would be figuring out a reliable way to define the identifier, without massively duplicating or requiring that something be registered somewhere. ChrisA
On Thu, May 20, 2021, 2:11 PM Chris Angelico
Probably the easiest way would be to have some kind of unique identifier (a fully-qualified name) that can be pickled, and then any time you attempt to construct a Sentinel with that identifier, it's guaranteed to return the same object.
Gosh, almost like a UUID :-). Actually, there's no reason my ._uuid attribute couldn't simply be the object id(). That's probably an improvement.
On 5/20/21 10:47 AM, David Mertz wrote:
On Thu, May 20, 2021 at 6:21 AM Tal Einat wrote:
I think it's worth preserving the idiom of comparing sentinels using `is`, as we do for `None` and other existing sentinel values. It's relatively easy to do, such as by using a single-value Enum or by using a class with a custom __new__.
This only works if:
a) Unpickling is within a single interpreter session
I don't understand. If I pickle a sentinel in session A, then unpickle it in session B, why wouldn't I get the "same" sentinel?
b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation
Why would you ever have a sentinel not always created? Hoping-to-learn-something-new'ly yrs, -- ~Ethan~
The scenario I'm thinking about is like this: (base) 84-tmp % python Python 3.9.1 (default, Dec 11 2020, 14:32:07) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
import pickle cheap_sentinel = object() id(cheap_sentinel) 140469343429632 pickle.dump(cheap_sentinel, open('sentinel.pkl', 'wb'))
(base) 85-tmp % python Python 3.9.1 (default, Dec 11 2020, 14:32:07) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
import pickle cheap_sentinel = object() id(cheap_sentinel) 139852505814016 id(pickle.load(open('sentinel.pkl', 'rb'))) 139852505813968
It would be pleasant if there were a way to make "cheap_sentinel" be the same thing—either by equality or by identity—betweeen those two runs of the interpreter. None or Ellipsis have that property, of course. So do, for example, integers, at least by equality if not identity (yes, of course, we might get identity by interning, but it's not guaranteed). On Thu, May 20, 2021 at 3:10 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 5/20/21 10:47 AM, David Mertz wrote:
On Thu, May 20, 2021 at 6:21 AM Tal Einat wrote:
I think it's worth preserving the idiom of comparing sentinels using `is`, as we do for `None` and other existing sentinel values. It's relatively easy to do, such as by using a single-value Enum or by using a class with a custom __new__.
This only works if:
a) Unpickling is within a single interpreter session
I don't understand. If I pickle a sentinel in session A, then unpickle it in session B, why wouldn't I get the "same" sentinel?
b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation
Why would you ever have a sentinel not always created?
Hoping-to-learn-something-new'ly yrs,
-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OLTV44EV... Code of Conduct: http://python.org/psf/codeofconduct/
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Thu, May 20, 2021 at 10:37 PM David Mertz <mertz@gnosis.cx> wrote:
The scenario I'm thinking about is like this:
(base) 84-tmp % python Python 3.9.1 (default, Dec 11 2020, 14:32:07) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
import pickle cheap_sentinel = object() id(cheap_sentinel) 140469343429632 pickle.dump(cheap_sentinel, open('sentinel.pkl', 'wb'))
(base) 85-tmp % python Python 3.9.1 (default, Dec 11 2020, 14:32:07) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
import pickle cheap_sentinel = object() id(cheap_sentinel) 139852505814016 id(pickle.load(open('sentinel.pkl', 'rb'))) 139852505813968
It would be pleasant if there were a way to make "cheap_sentinel" be the same thing—either by equality or by identity—betweeen those two runs of the interpreter.
None or Ellipsis have that property, of course. So do, for example, integers, at least by equality if not identity (yes, of course, we might get identity by interning, but it's not guaranteed).
There are several ways of achieving this for other sentinel values, some of which have been mentioned earlier in this thread. Some examples are: 1. a value from a single-valued enum 2. a class object (not an instance) 3. a singleton class with a carefully implemented __new__ which always returns the same instance - Tal Einat
On Sat, 15 May 2021, 5:39 am Tal Einat, <taleinat@gmail.com> wrote: (snip useful feature summary) The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread.
I'd also personally prefer something simple, ideally without a new class or module.
The key advantage of a new base class is that it offers a way to communicate a shift in the recommended development idiom. "sentinel = object()" is entrenched enough that I would expect that only a "types.Sentinel" base class would stand a good chance of displacing it. Why the "types" module? I don't think this is important enough to be a builtin type, and the types module is cheap to import, doesn't bring in many transitive dependencies, and the purpose of the new base class would be defining custom sentinel types with various desirable properties. Cheers, Nick.
Since the subject is this, I will note that past week, I resorted twice to create an Enum with a single element, and then alias the element on the module namespace, so that it would work as a "polished" sentinel. So: import enum class Whatever(enum.Enum): EMPTY = "EMPTY" EMPTY = Whatever.EMPTY A "repeat yourself three times" - certainly not that nice. But its use was a bit off the ones listed here in a sense these are meant to be explicitly passed to some methods (rather, placed inside data structures) to indicate a desired behavior. On Fri, 14 May 2021 at 21:03, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Sat, 15 May 2021, 5:39 am Tal Einat, <taleinat@gmail.com> wrote:
(snip useful feature summary)
The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread.
I'd also personally prefer something simple, ideally without a new class or module.
The key advantage of a new base class is that it offers a way to communicate a shift in the recommended development idiom. "sentinel = object()" is entrenched enough that I would expect that only a "types.Sentinel" base class would stand a good chance of displacing it.
Why the "types" module? I don't think this is important enough to be a builtin type, and the types module is cheap to import, doesn't bring in many transitive dependencies, and the purpose of the new base class would be defining custom sentinel types with various desirable properties.
Cheers, Nick.
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/V67I4A6W... Code of Conduct: http://python.org/psf/codeofconduct/
I think that would be the primary motivating factor behind recommending Ellipsis, it’s already a builtin and we are not likely it to get another builtin singleton. Ever? But besides that “...” in a function signature, although maybe looking magical, does immediately call out to the reader that something special is happening here. And for beginners who might not know that three dots are an Ellipsis or what an Ellipsis even is, it would be fairly easy to internet search “three dots in a function signature” or even “python def ...” would probably be enough to surface an explanation if this became somewhat common.
I've created a poll on this subject: https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810 On Fri, May 14, 2021 at 10:33 PM Tal Einat <taleinat@gmail.com> wrote:
On Fri, May 14, 2021 at 12:45 PM Steve Dower <steve.dower@python.org> wrote:
On 14May2021 0622, micro codery wrote:
There was a discussion a while back ( a year or so?? ) on Python-ideas that introduced the idea of having more "sentinel-like" singletons in Python -- right now, we only have None.
Not quite true, we also have Ellipsis, which already has a nice repr that both reads easily and still follows the convention of eval(repr(x)) == x. It also is already safe from instantiation, survives pickle round-trip and is multi-thread safe. So long as you are not dealing with scientific projects, it seems a quick (if dirty) solution to having a sentinel that is not None.
I don't think using "..." to indicate "some currently unknown or unspecified value" is dirty at all, it seems perfectly consistent with how we use it in English (and indexing in scientific projects, for that matter, where it tends to imply "figure out the rest for me").
All that's really missing is some kind of endorsement (from python-dev, presumably in the docs) that it's okay to use it as a default parameter value. I can't think of any reason you'd need to accept Ellipsis as a *specified* value that wouldn't also apply to any other kind of shared sentinel.
I'll try to organize my thoughts a bit here. This is a bit long, welcome to skip to the final sentence for the "tl;dr".
Features one may want for a sentinel: 1. Unique from other objects 2. Globally unique, i.e. unique from other such sentinels (no consensus here) 3. Clear repr (the original subject of this thread) - significant since these often appear in function signatures 4. Survives pickling round-trip (I brought this up since I was bitten by this once, but others have mentioned that this is usually irrelevant) 5. Can be given a clear type signature (this was brought up on twitter[1]) - significant since without this nobody can add full type signatures even if they want to
The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is what I've been using for years, and I now think that it isn't good enough. This not having a nice repr is what started this thread.
I'd also personally prefer something simple, ideally without a new class or module.
There are several simple idioms using existing features that seem like good options, so let's review those:
1. Ellipsis, a.k.a. `...`. This has all of the features outlined above except #2. My main issue with using Ellipsis for this is that it could be surprising and confusing for devs first encountering such use, and could be relatively awkward to figure out.
2. An instance of a one-off class. dataclasses.MISSING is an example of this. It is defined thus:
class _MISSING: pass MISSING = _MISSING()
Besides failing #4 (surviving pickle round-trips), its repr isn't great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily overcome by implementing __repr__.
3. A one-off class:
class MISSING: pass
This has all of the above features except #5: having a clear type signature (since its type is type). Using a class as a value this way could be surprising, though. It's repr also isn't great: <class 'dataclasses._MISSING'>.
4. A value of an single-valued enum, for example (from [1]):
class _UNSET(enum.Enum): token = enum.auto()
This has all of the features above and is simple, just requiring a comment to explain what it is. It's repr is a bit awkward though:
repr(_UNSET.token) '<_UNSET.token: 1>'
All of these are in use by some developers, though not necessarily in the stdlib. None is perfect, though all are probably good enough. Since pickling is likely not relevant in most cases, I'm currently in favor of #2 making sure to implement a nice __repr__.
- Tal
IMO you should consider writing a PEP to enhance sentinels in Python, and maybe even provide a public API for sentinels in general. Victor
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis". If you don't like the name for this purpose, you can always define a constant (that won't fix the `repr`, obviously, but helps with source code readability). SENTINEL = ... I can't think of any case where I'd rather have my own custom sentinel, or need a special API for sentinels. Probably my fault, of course. Please enlighten me! Cheers, Luciano On Thu, May 20, 2021 at 8:35 AM Victor Stinner <vstinner@python.org> wrote:
IMO you should consider writing a PEP to enhance sentinels in Python, and maybe even provide a public API for sentinels in general.
Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EY6B6PRQ... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Thu, 20 May 2021 at 18:13, Luciano Ramalho <luciano@ramalho.org> wrote:
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis".
Personally, I'm quite tempted by the idea of using ellipsis. It just sort of feels reasonable (and in the context `def f(x, optional_arg=...)` it even looks pretty natural). But it nevertheless feels like a bit of an abuse - the original point of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it, even if it's relatively rare in everyday Python. So while I like the idea in principle, I'm mildly worried that it's not "the right thing to do". I can't put my ambivalence about the idea any more precisely than this, unfortunately. Paul
On 5/20/21 11:00 AM, Paul Moore wrote:
But it nevertheless feels like a bit of an abuse - the original point of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it,
Interesting -- do you know what ... means in that context? -- ~Ethan~
On Thu, May 20, 2021 at 3:03 PM Ethan Furman <ethan@stoneleaf.us> wrote:
But it nevertheless feels like a bit of an abuse - the original point of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it, Interesting -- do you know what ... means in that context?
In NumPy, the ellipsis means "fill in as many dimensions as needed (with full range)". So e.g., if I have a 5-D array, and I want just a portion from the first and last dimension (but everything from the middle ones), I can type: a[1:20:2, :, :, :, 3:5] But as a simplification, I can use the example given: a[1:20:2, ..., 3:5] This is particularly useful since in NumPy it is not uncommon to expand or contract the number of dimensions (often with some dimensions having only a span of 1). If you don't want to think about which version of the high-dimensional array you are working with (that might have been .flatten()'d, .squeeze()'d, or .expand_dims()'d), this is sometimes more expressive. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
But it nevertheless feels like a bit of an abuse - the original point
of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it, Interesting -- do you know what ... means in that context?
In NumPy, the ellipsis means "fill in as many dimensions as needed (with full range)".
I am in the same boat here in that until now I saw Ellipsis as a thing used by numpy and adjacent libraries in slicing. I don't think I have ever written a slice with "..." in it so please forgive any ignorance but wasn't the reason for having Ellipsis originally that None already had a special meaning to the slice object and a different sentinel was needed? In other words the normal case of a missing argument being None caused specific side effects that needed to be skipped, so a sentinel was created to distinguish these cases, just like we are discussing for other objects? It seems to me the current situation has more in common than not. Numpy could have created its own Sentinel object for its own use in these sliceses, just as it now creates its own Sentinel for NoValue, but somewhere along the way it was determined to be in the better interest of the community to have this object defined in the standard library so it could be shared by multiple scientific libraries without additional dependency burden, even though it had no use to CPython. I think there was perhaps a time when Ellipsis was so closely tied to the scientific libraries that recommending its use anywhere else would have been a poor idea, probably mostly because of the body of existing teaching around its use; but I also think that association is no longer as strong. "..." is being adopted across the typing landscape with several PEPs, such as 483, outlining their special meaning in annotations. In addition, stub files have since the beginning normalized on putting "..." as the function body despite already having the working conventions of `pass` `NotImplemented` or just a good old empty docstring """""" filling the same purpose in standard python files.
On Thu, 20 May 2021 at 20:06, Ethan Furman <ethan@stoneleaf.us> wrote:
On 5/20/21 11:00 AM, Paul Moore wrote:
But it nevertheless feels like a bit of an abuse - the original point of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it,
Interesting -- do you know what ... means in that context?
In general, it just means a.getitem((slice(1,20,2), Ellipsis, slice(3,5))), which has no specifically-defined meaning. In numpy, it means something along the lines of "broadcast along this axis" (I don't know the numpy terminology very well). Paul
On Thu, 2021-05-20 at 19:00 +0100, Paul Moore wrote:
On Thu, 20 May 2021 at 18:13, Luciano Ramalho <luciano@ramalho.org> wrote:
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis".
Personally, I'm quite tempted by the idea of using ellipsis. It just sort of feels reasonable (and in the context `def f(x, optional_arg=...)` it even looks pretty natural).
But it nevertheless feels like a bit of an abuse - the original point of ellipsis was for indexing, and in particular complex slices like a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand it, even if it's relatively rare in everyday Python. So while I like the idea in principle, I'm mildly worried that it's not "the right thing to do".
I can't put my ambivalence about the idea any more precisely than this, unfortunately.
In NumPy we use a "missing argument" sentinel currently. Mainly for things roughly like: def mean(arr, *, axis=np._NoValue): if not hasattr(arr, "mean"): # Not a duck that defines `mean`, coerce to ndarray: arr = np.asarray(arr) if axis is np._NoValue: return arr.mean() return arr.mean(axis=axis) This allows us to add new keyword arguments without breaking backward compatibility. I do not remember if we had particularly important reasons for not wanting to drop the default `None`, or it was just erring on the safe side. In any case, I tend to agree that `Ellipsis` should be considered "user-facing" value. And in the above code, we do not expect anyone to ever call `np.mean(something, axis=np._NoValue)` – its not even accessible – but if the value was `...` then I would expect users to be encouraged to write `np.mean(arr, axis=...)` in normal code. More importantly, I can think of a reasonable "meaning" for `axis=...`! In NumPy `axis=None` (default) returns a scalar, `axis=...` could return a 0-D array. This would borrow meanings that `Ellipsis` carries in indexing. [1] Cheers, Sebastian [1] In such a mental model, it would mean the same as `axis=range(arr.ndim)`. To be clear, NumPy doesn't do this, its just a plausible meaning if it has to continue to juggle scalars and 0-D arrays and wants to be "clearer" about it.
Paul _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZGMZRGHF... Code of Conduct: http://python.org/psf/codeofconduct/
On 20 May 2021, at 19:10, Luciano Ramalho <luciano@ramalho.org> wrote:
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis".
If you don't like the name for this purpose, you can always define a constant (that won't fix the `repr`, obviously, but helps with source code readability).
SENTINEL = ...
I can't think of any case where I'd rather have my own custom sentinel, or need a special API for sentinels. Probably my fault, of course. Please enlighten me!
One use case for a sentinel that is not a predefined (builtin) singleton is APIs where an arbitrary user specified value can be used. One example of this is the definition of dataclasses.field: dataclasses.field(*, default=MISSING, default_factory=MISSING, repr=True, hash=None, init=True, compare=True, metadata=None) Here the “default” and “default_factory” can be an arbitrary value, and any builtin singleton could be used. Hence the use of a custom module-private sentinel that cannot clash with values used by users of the module (unless those users poke at private details of the module, but then all bets are off anyway). That’s why I don’t particularly like the proposal of using Ellipsis as the sanctioned sentinel value. It would be weird at best that the default for a dataclass field can be any value, except for the builtin Ellipsis value. Ronald
Cheers,
Luciano
On Thu, May 20, 2021 at 8:35 AM Victor Stinner <vstinner@python.org> wrote:
IMO you should consider writing a PEP to enhance sentinels in Python, and maybe even provide a public API for sentinels in general.
Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EY6B6PRQ... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WNCFFIJH... Code of Conduct: http://python.org/psf/codeofconduct/
— Twitter / micro.blog: @ronaldoussoren Blog: https://blog.ronaldoussoren.net/
On 5/20/2021 3:24 PM, Ronald Oussoren via Python-Dev wrote:
On 20 May 2021, at 19:10, Luciano Ramalho <luciano@ramalho.org <mailto:luciano@ramalho.org>> wrote:
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis".
If you don't like the name for this purpose, you can always define a constant (that won't fix the `repr`, obviously, but helps with source code readability).
SENTINEL = ...
I can't think of any case where I'd rather have my own custom sentinel, or need a special API for sentinels. Probably my fault, of course. Please enlighten me!
One use case for a sentinel that is not a predefined (builtin) singleton is APIs where an arbitrary user specified value can be used.
One example of this is the definition of dataclasses.field:
|dataclasses.||field|(/*/, /default=MISSING/, /default_factory=MISSING/, /repr=True/, /hash=None/, /init=True/, /compare=True/, /metadata=None/)
Here the “default” and “default_factory” can be an arbitrary value, and any builtin singleton could be used. Hence the use of a custom module-private sentinel that cannot clash with values used by users of the module (unless those users poke at private details of the module, but then all bets are off anyway).
That’s why I don’t particularly like the proposal of using Ellipsis as the sanctioned sentinel value. It would be weird at best that the default for a dataclass field can be any value, except for the builtin Ellipsis value.
Completely agree. I'm opposed to Ellipsis as a sentinel for this reason, at least for dataclasses. I can easily see wanting to store an Ellipsis in a field of a dataclass that's describing a function's parameters. And I can even see it being the default= value. Not so much default_factory=, but they may as well be the same. Eric
On 21. 05. 21 3:23, Eric V. Smith wrote:
On 5/20/2021 3:24 PM, Ronald Oussoren via Python-Dev wrote:
On 20 May 2021, at 19:10, Luciano Ramalho <luciano@ramalho.org <mailto:luciano@ramalho.org>> wrote:
I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is not a good sentinel. It's a pickable singleton testable with `is`, readily available, and extremely unlikely to appear in a data stream. Its repr is "Ellipsis".
If you don't like the name for this purpose, you can always define a constant (that won't fix the `repr`, obviously, but helps with source code readability).
SENTINEL = ...
I can't think of any case where I'd rather have my own custom sentinel, or need a special API for sentinels. Probably my fault, of course. Please enlighten me!
One use case for a sentinel that is not a predefined (builtin) singleton is APIs where an arbitrary user specified value can be used.
One example of this is the definition of dataclasses.field:
|dataclasses.||field|(/*/, /default=MISSING/, /default_factory=MISSING/, /repr=True/, /hash=None/, /init=True/, /compare=True/, /metadata=None/)
Here the “default” and “default_factory” can be an arbitrary value, and any builtin singleton could be used. Hence the use of a custom module-private sentinel that cannot clash with values used by users of the module (unless those users poke at private details of the module, but then all bets are off anyway).
That’s why I don’t particularly like the proposal of using Ellipsis as the sanctioned sentinel value. It would be weird at best that the default for a dataclass field can be any value, except for the builtin Ellipsis value.
Completely agree. I'm opposed to Ellipsis as a sentinel for this reason, at least for dataclasses. I can easily see wanting to store an Ellipsis in a field of a dataclass that's describing a function's parameters. And I can even see it being the default= value. Not so much default_factory=, but they may as well be the same.
And this argument also works for any other single value. Including the original None. (It just might not be obvious at first, before that single value starts being used in lots of different contexts.)
On 5/21/2021 9:36 AM, Petr Viktorin wrote:
On 21. 05. 21 3:23, Eric V. Smith wrote:
On 5/20/2021 3:24 PM, Ronald Oussoren via Python-Dev wrote:
One example of this is the definition of dataclasses.field:
|dataclasses.||field|(/*/, /default=MISSING/, /default_factory=MISSING/, /repr=True/, /hash=None/, /init=True/, /compare=True/, /metadata=None/)
Completely agree. I'm opposed to Ellipsis as a sentinel for this reason, at least for dataclasses. I can easily see wanting to store an Ellipsis in a field of a dataclass that's describing a function's parameters. And I can even see it being the default= value. Not so much default_factory=, but they may as well be the same.
And this argument also works for any other single value. Including the original None.
(It just might not be obvious at first, before that single value starts being used in lots of different contexts.)
I think it's a fairly obvious case, and a legitimate one to acknowledge (the array one less so - we're talking about a "missing parameter" sentinel, so you ought to be testing for it immediately and replacing with your preferred/secret default value, not using it for indexing without even checking it). In the example above, it's fairly easy to pass "lambda: ..." as the default_factory to work around it, and besides, the existence of rare edge cases doesn't mean you have to force everyone into acting like they're a rare edge case. All the other situations where we want arguments with unspecified default values can use ..., and the few cases where ... is a valid value (semantically, for the API, not syntactically) can spend the time figuring out a different API design. Cheers, Steve
I was attracted to Python in 1998 because it seemed designed to make the simple cases simple, and the hard cases possible. My personal takeaway from this discussion: I will continue to advocate for the use of Ellipsis as a sentinel in the *many* cases where it is suitable. For the hard cases, I will read the upcoming 46-page "PEP 973: Parameterized Sentinel Factory Factory API" ;-) Cheers, Luciano -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Fri, May 21, 2021 at 5:33 PM Luciano Ramalho <luciano@ramalho.org> wrote:
For the hard cases, I will read the upcoming 46-page "PEP 973: Parameterized Sentinel Factory Factory API" ;-)
I put up an early draft of a PEP on a branch in the PEPs repo: https://github.com/python/peps/blob/sentinels/pep-9999.rst (Note: This link will break once the temporary branch is deleted.) I wrote it to summarize the discussions, organize my thoughts and explore the options. I stopped working on it late at night and sent a link to a few people to get some opinions. I didn’t intend to make it public yet, but it was noticed and replied to on the discuss.python.org thread where I put up the poll [1], so the cat is out of the proverbial bag now… Luciano, your wish is granted! ;) - Tal Einat [1] https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/
On Sun, May 23, 2021 at 3:37 AM Tal Einat <taleinat@gmail.com> wrote:
I put up an early draft of a PEP on a branch in the PEPs repo: https://github.com/python/peps/blob/sentinels/pep-9999.rst
Thanks for that PEP, Tal. Good ideas and recap there. I think repr= should have a default: the name of the class within <>: <NotGiven>. Sentinels don't have state or any other data besides a name, so I would prefer not to force users to create a class just so they can instantiate it. Why not just this? NotGiven = sentinel('<NotGiven>') In this case it's harder to provide a good default repr. On the other hand, if the user must create a class, the class itself should be the sentinel. Class objects are already singletons, so that makes sense. Here is a possible class-based API: class NotGiven(Sentinel): pass That's it. Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>. Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method. What do you think? Cheers, Luciano
(Note: This link will break once the temporary branch is deleted.)
I wrote it to summarize the discussions, organize my thoughts and explore the options. I stopped working on it late at night and sent a link to a few people to get some opinions. I didn’t intend to make it public yet, but it was noticed and replied to on the discuss.python.org thread where I put up the poll [1], so the cat is out of the proverbial bag now…
Luciano, your wish is granted! ;) - Tal Einat
[1] https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
Sorry about my detour into the rejected idea of a factory function. But how about this class-based API? class NotGiven(Sentinel): pass Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>. Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method. Cheers, Luciano
(Note: This link will break once the temporary branch is deleted.)
I wrote it to summarize the discussions, organize my thoughts and explore the options. I stopped working on it late at night and sent a link to a few people to get some opinions. I didn’t intend to make it public yet, but it was noticed and replied to on the discuss.python.org thread where I put up the poll [1], so the cat is out of the proverbial bag now…
Luciano, your wish is granted! ;) - Tal Einat
[1] https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On 2021-05-24 01:37, Luciano Ramalho wrote:
Sorry about my detour into the rejected idea of a factory function.
But how about this class-based API?
class NotGiven(Sentinel): pass
Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
The repr of other singletons are the names of those singletons, eg. "None", so why "<NotGiven>" instead of "NotGiven"?
Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method.
On Mon, May 24, 2021 at 4:10 AM MRAB <python@mrabarnett.plus.com> wrote:
On 2021-05-24 01:37, Luciano Ramalho wrote:
Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
The repr of other singletons are the names of those singletons, eg. "None", so why "<NotGiven>" instead of "NotGiven"?
Yea, that's up in the air. The common suggestions are either "NotGiven", "<NotGiven>" or "mymodule.NotGiven". The first makes sense for builtins like None and Ellipses, but I'm not sure a function signature like foo(bar=NotGiven) is very clear. With the factory function pattern there's no need for a default, so this may become a non-issue, and I may remove the recommendation for which form to use. - Tal
Here is a simple implementation of that Sentinel class: https://github.com/fluentpython/example-code-2e/blob/master/25-class-metapro... Tests in the same directory. That's not a real package yet, just a couple of super simple examples I may use in Fluent Python 2e. Cheers, Luciano On Sun, May 23, 2021 at 9:37 PM Luciano Ramalho <luciano@ramalho.org> wrote:
Sorry about my detour into the rejected idea of a factory function.
But how about this class-based API?
class NotGiven(Sentinel): pass
Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method.
Cheers,
Luciano
(Note: This link will break once the temporary branch is deleted.)
I wrote it to summarize the discussions, organize my thoughts and explore the options. I stopped working on it late at night and sent a link to a few people to get some opinions. I didn’t intend to make it public yet, but it was noticed and replied to on the discuss.python.org thread where I put up the poll [1], so the cat is out of the proverbial bag now…
Luciano, your wish is granted! ;) - Tal Einat
[1] https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
Thanks Tal for writing this up. A couple comments: 1) “Add a single new sentinel value, e.g. MISSING or Sentinel” (under rejected) I was one of the proponent of that -- but not as an alternative to having a stadardars way to create unique sentinels, but as an addition. That's kind of orthogonal to the PEP, but it would still be nice to have it at the same time. Why? There was a fair bit of discussion as to why you would not want to require everyone to use MISSING (dataclasses, in particular), but I still think better for the readability of everyone's code for the common case(s?) to use the same singleton, if they can. 2) couldn't a factory function simply return a sentinel subclass? Maybe I'm missing something, but I can't see why that would require stack frame inspection. But frankly Luciano's idea of a base class that can be subclassed seems the most startightford to me. -CHB
On Mon, May 24, 2021 at 6:04 AM Christopher Barker <pythonchb@gmail.com> wrote:
1) “Add a single new sentinel value, e.g. MISSING or Sentinel” (under rejected)
I was one of the proponent of that -- but not as an alternative to having a stadardars way to create unique sentinels, but as an addition. That's kind of orthogonal to the PEP, but it would still be nice to have it at the same time.
Why? There was a fair bit of discussion as to why you would not want to require everyone to use MISSING (dataclasses, in particular), but I still think better for the readability of everyone's code for the common case(s?) to use the same singleton, if they can.
The way I see it, there are several clear motivations to implement one of these and make that the recommended way of defining sentinels. The further benefit of having both is less clear to me. Also, there's this in the Zen of Python: "There should be one-- and preferably only one --obvious way to do it." This strengthens my feeling that having two recommended ways of defining sentinel values, in addition to the existing None, would be undesirable.
2) couldn't a factory function simply return a sentinel subclass? Maybe I'm missing something, but I can't see why that would require stack frame inspection.
It seems better for each sentinel to have its own class, which makes it possible to write strict type signatures and allows better static code analysis. It also makes handling copying and unpickling simple.
But frankly Luciano's idea of a base class that can be subclassed seems the most startightford to me.
Yes, and it's what I originally suggested near the beginning of this thread :) - Tal
On Mon, May 24, 2021 at 11:44 AM Tal Einat <taleinat@gmail.com> wrote:
But frankly Luciano's idea of a base class that can be subclassed seems the most startightford to me.
Yes, and it's what I originally suggested near the beginning of this thread :)
I am sorry to have missed your previous e-mail with a base class for sentinels, @Tal. I support that idea. In fact, if we have a SentinelMeta metaclass, and a Sentinel base class built from SentinelMeta, the Sentinel class can be used directly as a sentinel without the need for subclassing—if the application does not require a custom sentinel. If it does, then the user can subclass Sentinel. The SentinelMeta could be private, to discourage misuse. This is my implementation, after learning from @Tal's code: https://github.com/fluentpython/example-code-2e/blob/master/25-class-metapro... Since having a Sentinel base class is desirable for ease of use, I think it is simpler to code the __new__ method in it, instead of coding metaclass logic to inject __new__ in the class namespace. Best, Luciano -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
Thanks for this, Luciano! My main issue with using class objects is that such use of them would be unusual and potentially confusing. I think that is important in general, and doubly so for something I suggest adding to the stdlib. Additionally, I very much would like for each sentinel object to have its own dedicated class, to allow writing strict function type signatures. Once one adds that to a meta-class or class decorator, the complexity of the implementation approaches what I currently have for a sentinel() function. At that point, I can't see a clear advantage for such an approach. - Tal On Mon, May 24, 2021 at 7:31 PM Luciano Ramalho <luciano@ramalho.org> wrote:
But frankly Luciano's idea of a base class that can be subclassed seems the most startightford to me.
Yes, and it's what I originally suggested near the beginning of this
On Mon, May 24, 2021 at 11:44 AM Tal Einat <taleinat@gmail.com> wrote: thread :)
I am sorry to have missed your previous e-mail with a base class for sentinels, @Tal. I support that idea.
In fact, if we have a SentinelMeta metaclass, and a Sentinel base class built from SentinelMeta, the Sentinel class can be used directly as a sentinel without the need for subclassing—if the application does not require a custom sentinel. If it does, then the user can subclass Sentinel.
The SentinelMeta could be private, to discourage misuse.
This is my implementation, after learning from @Tal's code:
https://github.com/fluentpython/example-code-2e/blob/master/25-class-metapro...
Since having a Sentinel base class is desirable for ease of use, I think it is simpler to code the __new__ method in it, instead of coding metaclass logic to inject __new__ in the class namespace.
Best,
Luciano
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Mon, May 24, 2021 at 3:30 AM Luciano Ramalho <luciano@ramalho.org> wrote:
On Sun, May 23, 2021 at 3:37 AM Tal Einat <taleinat@gmail.com> wrote:
I put up an early draft of a PEP on a branch in the PEPs repo: https://github.com/python/peps/blob/sentinels/pep-9999.rst
Thanks for that PEP, Tal. Good ideas and recap there.
I think repr= should have a default: the name of the class within <>: <NotGiven>.
Sentinels don't have state or any other data besides a name, so I would prefer not to force users to create a class just so they can instantiate it.
Why not just this?
NotGiven = sentinel('<NotGiven>')
I'm seriously considering that now. The issues I ran into with this approach are perhaps not actually problematic.
On the other hand, if the user must create a class, the class itself should be the sentinel. Class objects are already singletons, so that makes sense.
Here is a possible class-based API:
class NotGiven(Sentinel): pass
That's it. Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method.
What do you think?
One issue with that is that such sentinels don't have their own class, so you can't write a strict type signature, such as `Union[str, NotGivenType]`. Another issue is that having these objects be classes, rather than normal instances of classes, could be surprising and confusing. For those two reasons, for now, I think generating a unique object with its own unique class is preferable.
Sorry about my detour into the rejected idea of a factory function.
Please don't apologize! I put those ideas in the "Rejected Ideas" section mostly to have them written down with a summary of the considerations related to them. They shouldn't be considered finally rejected unless and until the PEP is finished and accepted. - Tal
Hello, and thanks for the PEP, I feel like the 3-lines declaration of a new sentinel would discourage a bit its adoption compared to just "sentinel = object()" From what I understand from the PEP, if new classes are defined inside the closure of a factory function, some Python implementations would have trouble copying/pickling them? Would it be doable to have a single Sentinel class, whose instances store their representation and some autogenerated UUID, and which automatically return internally stored singletons (depending on this UUID) when called multiple times or unpickled ? This would require some __new__() and unpickling magic, but nothing too CPython-specific (or am I missing something?). regards, Pascal Le 24/05/2021 à 16:28, Tal Einat a écrit :
On Mon, May 24, 2021 at 3:30 AM Luciano Ramalho <luciano@ramalho.org> wrote:
On Sun, May 23, 2021 at 3:37 AM Tal Einat <taleinat@gmail.com> wrote:
I put up an early draft of a PEP on a branch in the PEPs repo: https://github.com/python/peps/blob/sentinels/pep-9999.rst Thanks for that PEP, Tal. Good ideas and recap there.
I think repr= should have a default: the name of the class within <>: <NotGiven>.
Sentinels don't have state or any other data besides a name, so I would prefer not to force users to create a class just so they can instantiate it.
Why not just this?
NotGiven = sentinel('<NotGiven>') I'm seriously considering that now. The issues I ran into with this approach are perhaps not actually problematic.
On the other hand, if the user must create a class, the class itself should be the sentinel. Class objects are already singletons, so that makes sense.
Here is a possible class-based API:
class NotGiven(Sentinel): pass
That's it. Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method.
What do you think? One issue with that is that such sentinels don't have their own class, so you can't write a strict type signature, such as `Union[str, NotGivenType]`.
Another issue is that having these objects be classes, rather than normal instances of classes, could be surprising and confusing.
For those two reasons, for now, I think generating a unique object with its own unique class is preferable.
Sorry about my detour into the rejected idea of a factory function. Please don't apologize! I put those ideas in the "Rejected Ideas" section mostly to have them written down with a summary of the considerations related to them. They shouldn't be considered finally rejected unless and until the PEP is finished and accepted.
- Tal _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HL74JC3O... Code of Conduct: http://python.org/psf/codeofconduct/ .
On Tue, May 25, 2021 at 1:08 PM Pascal Chambon <pythoniks@gmail.com> wrote:
I feel like the 3-lines declaration of a new sentinel would discourage a bit its adoption compared to just "sentinel = object()"
How about one line? class Missing(Sentinel): pass And avoiding a lot of complications? https://github.com/fluentpython/example-code-2e/blob/master/25-class-metapro... Cheers, Luciano -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Tue, May 25, 2021 at 11:25 AM Pascal Chambon <pythoniks@gmail.com> wrote:
Hello, and thanks for the PEP,
I feel like the 3-lines declaration of a new sentinel would discourage a bit its adoption compared to just "sentinel = object()"
I now tend to agree. The new version of the draft PEP proposes a simpler interface: NotGiven = sentinel('NotGiven')
From what I understand from the PEP, if new classes are defined inside the closure of a factory function, some Python implementations would have trouble copying/pickling them?
The reference implementations I propose take care of this. I'll make sure that everything works in popular alternate implementations such as PyPy.
Would it be doable to have a single Sentinel class, whose instances store their representation and some autogenerated UUID, and which automatically return internally stored singletons (depending on this UUID) when called multiple times or unpickled ? This would require some __new__() and unpickling magic, but nothing too CPython-specific (or am I missing something?).
That's certainly doable, and I believe that there are existing implementations of sentinels that use this method. But since classes are singletons, and it's simple to make a class that always returns the same object, there's no need for setting a UUID and implementing custom pickling logic as you suggest. Another drawback of this approach is that each sentinel value wouldn't have its own dedicated type. - Tal
Following the continued discussion, I'm currently in favor of a simple interface using a factory function, i.e. NotGiven = sentinel('NotGiven'). I've created a new GitHub repo with a reference implementation. It also includes a second version of the draft PEP, addresses some of the additional points brought up in the latest parts of this discussion. https://github.com/taleinat/python-stdlib-sentinels - Tal On Mon, May 24, 2021 at 5:28 PM Tal Einat <taleinat@gmail.com> wrote:
On Mon, May 24, 2021 at 3:30 AM Luciano Ramalho <luciano@ramalho.org> wrote:
On Sun, May 23, 2021 at 3:37 AM Tal Einat <taleinat@gmail.com> wrote:
I put up an early draft of a PEP on a branch in the PEPs repo: https://github.com/python/peps/blob/sentinels/pep-9999.rst
Thanks for that PEP, Tal. Good ideas and recap there.
I think repr= should have a default: the name of the class within <>: <NotGiven>.
Sentinels don't have state or any other data besides a name, so I would prefer not to force users to create a class just so they can instantiate it.
Why not just this?
NotGiven = sentinel('<NotGiven>')
I'm seriously considering that now. The issues I ran into with this approach are perhaps not actually problematic.
On the other hand, if the user must create a class, the class itself should be the sentinel. Class objects are already singletons, so that makes sense.
Here is a possible class-based API:
class NotGiven(Sentinel): pass
That's it. Now I can use NotGiven as the sentinel, and its default repr is <NotGiven>.
Behind the scenes we can have a SentinelMeta metaclass with all the magic that could be required--including the default __repr__ method.
What do you think?
One issue with that is that such sentinels don't have their own class, so you can't write a strict type signature, such as `Union[str, NotGivenType]`.
Another issue is that having these objects be classes, rather than normal instances of classes, could be surprising and confusing.
For those two reasons, for now, I think generating a unique object with its own unique class is preferable.
Sorry about my detour into the rejected idea of a factory function.
Please don't apologize! I put those ideas in the "Rejected Ideas" section mostly to have them written down with a summary of the considerations related to them. They shouldn't be considered finally rejected unless and until the PEP is finished and accepted.
- Tal
Hi Tal, Would it make sense to have an unique singleton for such sentinel, a built-in singleton like None or Ellipsis? I propose the name "Sentinel". Sentinel would be similar to None, but the main property would be that "Sentinel is None" is false :-) The stdlib contains tons of sentinels: * _collections_abc: __marker__ * cgitb.__UNDEF__ * configparser: _UNSET * dataclasses: _HAS_DEFAULT_FACTORY, MISSING, KW_ONLY * datetime.timezone._Omitted * fnmatch.translate() STAR * functools.lru_cache.sentinel (each @lru_cache creates its own sentinel object) * functools._NOT_FOUND * heapq: temporary sentinel in nsmallest() and nlargest() * inspect._sentinel * inspect._signature_fromstr() invalid * plistlib._undefined * runpy._ModifiedArgv0._sentinel * sched: _sentinel * traceback: _sentinel There are different but similar use cases: * Optional parameter: distinguish between func() and func(arg=value), a sentinel is useful to distinguish func() from func(arg=None) * Look into a data structure for a value and store the result in a value, distinguish if 'result' variable was set ("result is not None" doesn't work since None is a value). Quick example: "missing = object(); tmsg = self._catalog.get(message, missing); if tmsg is missing: ..." Special cases: * dataclases._EMPTY_METADATA = types.MappingProxyType({}) * string._sentinel_dict = {} * enum: _auto_null = object() Victor On Thu, May 13, 2021 at 7:40 PM Tal Einat <taleinat@gmail.com> wrote:
On Thu, May 13, 2021 at 7:44 PM Ethan Furman <ethan@stoneleaf.us> wrote:
Consider me complaining. ;-)
+1
An actual Sentinel class would be helpful:
>>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ...
>>> MISSING = Sentinel('MISSING') >>> MISSING MISSING
>>> implicit = Sentinel('<implicit>') >>> implicit <implicit>
Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr:
class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated')
class MISSING(Sentinel): pass
- Tal _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/URFRF634... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
On 14. 05. 21 10:55, Victor Stinner wrote:
Hi Tal,
Would it make sense to have an unique singleton for such sentinel, a built-in singleton like None or Ellipsis? I propose the name "Sentinel".
Sentinel would be similar to None, but the main property would be that "Sentinel is None" is false :-)
If you need your Sentinel to be different from one particular sentinel (None), you'll usually want it to be different from all the other ones as well. A sentinel for an optional parameter shouldn't really be used at all outside of the function it's defined for. That's why it's usually defined as a private module-level variable. Perhaps it would be beneficial to provide a common base class or factory, so we get a good repr. But I don't think another common value like None and Ellipsis would do much good.
The stdlib contains tons of sentinels:
* _collections_abc: __marker__ * cgitb.__UNDEF__ * configparser: _UNSET * dataclasses: _HAS_DEFAULT_FACTORY, MISSING, KW_ONLY * datetime.timezone._Omitted * fnmatch.translate() STAR * functools.lru_cache.sentinel (each @lru_cache creates its own sentinel object) * functools._NOT_FOUND * heapq: temporary sentinel in nsmallest() and nlargest() * inspect._sentinel * inspect._signature_fromstr() invalid * plistlib._undefined * runpy._ModifiedArgv0._sentinel * sched: _sentinel * traceback: _sentinel
There are different but similar use cases:
* Optional parameter: distinguish between func() and func(arg=value), a sentinel is useful to distinguish func() from func(arg=None) * Look into a data structure for a value and store the result in a value, distinguish if 'result' variable was set ("result is not None" doesn't work since None is a value). Quick example: "missing = object(); tmsg = self._catalog.get(message, missing); if tmsg is missing: ..."
Special cases:
* dataclases._EMPTY_METADATA = types.MappingProxyType({}) * string._sentinel_dict = {} * enum: _auto_null = object()
Victor
On Thu, May 13, 2021 at 7:40 PM Tal Einat <taleinat@gmail.com> wrote:
On Thu, May 13, 2021 at 7:44 PM Ethan Furman <ethan@stoneleaf.us> wrote:
Consider me complaining. ;-)
+1
An actual Sentinel class would be helpful:
>>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ...
>>> MISSING = Sentinel('MISSING') >>> MISSING MISSING
>>> implicit = Sentinel('<implicit>') >>> implicit <implicit>
Here is my suggestion (also posted on the related bpo-44123), which is also simple, ensures a single instance is used, even considering multi-threading and pickling, and has a better repr:
class Sentinel: def __new__(cls, *args, **kwargs): raise TypeError(f'{cls.__qualname__} cannot be instantiated')
class MISSING(Sentinel): pass
- Tal _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/URFRF634... Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, May 14, 2021 at 7:31 PM Petr Viktorin <encukou@gmail.com> wrote:
Perhaps it would be beneficial to provide a common base class or factory, so we get a good repr. But I don't think another common value like None and Ellipsis would do much good.
Agreed - I think Sentinel would make a great class, from which you instantiate purpose-specific sentinels. But maybe there really needs to be a way to NOT specify an argument, and to find out that an argument wasn't specified? Defaulting is only one way to handle it. Consider: def truly_optional_arg(x, y, *z): """Two mandatory args, and then an optional one""" if z: print("Got an extra arg") pass_arg = input("Pass the arg? ") == "y" truly_optional_arg(10, 20, *([30] if pass_arg else [])) It's horrifically ugly, but it really and truly does/doesn't pass that argument, and it really and truly detects whether one was passed. To make that sort of thing actually viable, there'd need to be some sort of language support; maybe something where the local name would start out unbound, but testing for a local's boundness is a clunky try/except, so that might also need some support. Do we ever really need the ability to pass a specific sentinel to a function, or are we actually looking for a way to say "and don't pass this argument"? ChrisA
On May 14, 2021, at 02:38, Chris Angelico <rosuav@gmail.com> wrote:
Do we ever really need the ability to pass a specific sentinel to a function, or are we actually looking for a way to say "and don't pass this argument”?
Very often, that’s the case. Such a “it’s okay to not pass this argument” construct would have to work with the Optional type too. The other use case I have for a special case single use singleton is for dict.get(), i.e. missing = object() value = somedict.get(‘key’, missing) if value is missing: # It ain’t there. -Barry
On Sat, May 15, 2021 at 2:04 AM Barry Warsaw <barry@python.org> wrote:
On May 14, 2021, at 02:38, Chris Angelico <rosuav@gmail.com> wrote:
Do we ever really need the ability to pass a specific sentinel to a function, or are we actually looking for a way to say "and don't pass this argument”?
Very often, that’s the case. Such a “it’s okay to not pass this argument” construct would have to work with the Optional type too.
What I mean is: how often do you actually need to pass the specific sentinel, as opposed to the normal construct of asking if the argument was or wasn't passed? Eg if you have a function like this: _sentinel = object() def func(x, y=_sentinel): if y is _sentinel: ... else: ... Would you ever call it like this: func(42, _sentinel) ? Because if there's no reason to ever pass the sentinel, then it's nothing more than an implementation detail for the concept of "was this argument passed?", and that's exactly what I'm asking about.
The other use case I have for a special case single use singleton is for dict.get(), i.e.
missing = object() value = somedict.get(‘key’, missing) if value is missing: # It ain’t there.
I'd write that one with a try/except instead. The whole point of get() is to provide the missing value. try: value = somedict['key'] except KeyError: # It ain't there. No sentinel needed. Same number of lines. ChrisA
On 5/13/2021 12:41 PM, Ethan Furman wrote:
On 5/13/21 2:15 AM, Irit Katriel via Python-Dev wrote:
>>> help(traceback.print_exception) Help on function print_exception in module traceback:
print_exception(exc, /, value=<object object at 0x000002825DF09650>, tb=<object object at 0x000002825DF09650>, limit=None, file=None, chain=True)
On 5/13/21 5:37 AM, Eric V. Smith wrote:
The help looks like:
field(*, default=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, default_factory=<dataclasses._MISSING_TYPE object at 0x6fffffe46610>, init=True, repr=True, hash=None, compare=True, metadata=None)
None of this is particularly awesome, but no one has complained about it yet.
Consider me complaining. ;-)
Your complaint is hereby noted!
Looks to me like the default repr for the sentinels is making those helps much less helpful by showing totally irrelevant information and cluttering up the screen making it harder to see the actually useful bits.
An actual Sentinel class would be helpful:
>>> class Sentinel: ... def __init__(self, repr): ... self.repr = repr ... def __repr__(self): ... return self.repr ...
>>> MISSING = Sentinel('MISSING') >>> MISSING MISSING
dataclasses.py actually has similar code, but for some reason I guess it got missed for MISSING (ha!).
Naturally, since sentinels are symbolic names, I think it should go into the enum module. ;-) Although I will concede that we could just put those five lines into the modules that need it.
Yeah, it's probably not worth dataclasses importing enum just to get that functionality. Eric
On 13May2021 1248, Petr Viktorin wrote:
On 13. 05. 21 11:45, Antoine Pitrou wrote:
Le 13/05/2021 à 11:40, Irit Katriel a écrit :
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org <mailto:antoine@python.org>> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>.
"<derived>" makes think of a derived class, and leaves me confused. "<implicit>" is a bit better, but doesn't clearly say what the default value is, either. So in all cases I have to read the docstring in addition to the function signature.
Is <default> the term you're looking for?
Perhaps <unspecified> or <missing>? Cheers, Steve
On Thu, 13 May 2021 13:44:54 +0100 Steve Dower <steve.dower@python.org> wrote:
On 13May2021 1248, Petr Viktorin wrote:
On 13. 05. 21 11:45, Antoine Pitrou wrote:
Le 13/05/2021 à 11:40, Irit Katriel a écrit :
On Thu, May 13, 2021 at 10:28 AM Antoine Pitrou <antoine@python.org <mailto:antoine@python.org>> wrote:
I agree that <optional> is a reasonable spelling.
I initially suggested <optional>, but now I'm not sure because it doesn't indicate what happens when you don't provide it (as in, what is the default value). So now I'm with <derived> or <implicit>.
"<derived>" makes think of a derived class, and leaves me confused. "<implicit>" is a bit better, but doesn't clearly say what the default value is, either. So in all cases I have to read the docstring in addition to the function signature.
Is <default> the term you're looking for?
Perhaps <unspecified> or <missing>?
Now that I read more about the specific use case, though, I think "<implicit>" really describes it accurately. It's not that the information is missing, it's that it's already implied in another argument. Quoting the documentation: """Since Python 3.10, instead of passing value and tb, an exception object can be passed as the first argument.""" (meaning the traceback is implicitly gotten from the exception object which is passed as first argument) Regards Antoine.
participants (22)
-
Antoine Pitrou -
Barry Warsaw -
Chris Angelico -
Christopher Barker -
David Mertz -
Eric V. Smith -
Ethan Furman -
Irit Katriel -
Joao S. O. Bueno -
Larry Hastings -
Luciano Ramalho -
micro codery -
MRAB -
Nick Coghlan -
Pascal Chambon -
Paul Moore -
Petr Viktorin -
Ronald Oussoren -
Sebastian Berg -
Steve Dower -
Tal Einat -
Victor Stinner