Mailman 3 Enum._convert should change __repr__ and/or __str__ to use module name instead of class name - Python-Dev

Enum._convert should change repr and/or str to use module name instead of class name

Ethan Furman

26 Mar 2020 26 Mar '20

4:54 a.m.

Serhiy had the idea of having Enum._convert also modify the __str__ and __repr__ of newly created enumerations to display the module name instead of the enumeration name (https://bugs.python.org/msg325007): --> socket.AF_UNIX ==> --> print(socket.AF_UNIX) AddressFamily.AF_UNIX ==> socket.AF_UNIX Thoughts? -- ~Ethan~

Show replies by date

Cameron Simpson

26 Mar 26 Mar

7:08 a.m.

On 25Mar2020 12:54, Ethan Furman wrote:

...

Serhiy had the idea of having Enum._convert also modify the __str__ and __repr__ of newly created enumerations to display the module name instead of the enumeration name (https://bugs.python.org/msg325007):

--> socket.AF_UNIX ==>

--> print(socket.AF_UNIX) AddressFamily.AF_UNIX ==> socket.AF_UNIX

Thoughts?

I'm uncomfortable. The socket module is something of a special case because its enums come from the C library constants, which therefore have nice global names like "AF_UNIX" because C is one big namespace; you can infer that they are address family constants from their prefix. I think a more "Python normal" module might have multiple enum classes, maybe with overlapping names. Invented but plausible example: class ANSIColourNames(enum): NORMAL = 0 REVERSE = 7 UNDERLINE = 4 BOLD = 1 BLACK = 30 RED = 31 ... and another: class SpecialNonANSITerminalCOlours(enum): NORMAL = 0 REVERSE = 15 RED = 3 ... i.e. different classes with the same purpose specific enum names inside. If I only got a module name back from __str__ I'd be underwhelmed. I think I'd at least like the behaviour switchable in some way. Cheers, Cameron Simpson

Serhiy Storchaka

7:35 a.m.

26.03.20 00:08, Cameron Simpson пише:

...

I think a more "Python normal" module might have multiple enum classes, maybe with overlapping names.

Do you have any examples of more "Python normal" modules? We discuss here the behavior of the private method Enum._convert_() used exclusively in the stdlib.

Cameron Simpson

8:04 a.m.

On 26Mar2020 00:35, Serhiy Storchaka wrote:

...

26.03.20 00:08, Cameron Simpson пише:

...
I think a more "Python normal" module might have multiple enum classes, maybe with overlapping names.

Do you have any examples of more "Python normal" modules?

Unfortunately no because I am not yet using enums as much as I should.

...

We discuss here the behavior of the private method Enum._convert_() used exclusively in the stdlib.

My concern is basicly that in normal Python usage, the names within a class (and thus the names in an enum) are named to be distinct within the class namespace. So I would not expect the names for particular values to have extra qualification (eg I expect them to be like the "RED" in my two class example - getting "module_name.RED" doesn't inform the user about which class is in use, and since they might reasonably have different concrete values like 4 versus 7 that seems unfortunate). I agree that having the enums from the socket module just recite "socket.AF_UNIX" would be nice. So if those enums overrode the Enum._convert_() method I see no reason to object. However, that only works because those names mirror the C library names, which are already globally unique. So my discomfort is present only if there were a general change affecting all enums, rather than a socket-module-specific change affecting just the enums from the socket module. Cheers, Cameron Simpson

Serhiy Storchaka

5:51 p.m.

26.03.20 01:04, Cameron Simpson пише:

...

My concern is basicly that in normal Python usage, the names within a class (and thus the names in an enum) are named to be distinct within the class namespace. So I would not expect the names for particular values to have extra qualification (eg I expect them to be like the "RED" in my two class example - getting "module_name.RED" doesn't inform the user about which class is in use, and since they might reasonably have different concrete values like 4 versus 7 that seems unfortunate).

module_name.RED can have only one value. It cannot be 4 and 7 at the same time. If you have an example of conflict, please report it on the bug tracker.

...

So my discomfort is present only if there were a general change affecting all enums, rather than a socket-module-specific change affecting just the enums from the socket module.

No, it is not a socket-module-specific change. No, it is not a general change affecting all enum. It is only about Enum._convert_().

Ivan Pozdeev

8:35 a.m.

I'm skeptical about anything that hides an object's "true nature": this is a major landmine in diagnostics because it lies to you about what you are looking at and where to look for its implementation. E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module. On 25.03.2020 22:54, Ethan Furman wrote:

...

Serhiy had the idea of having Enum._convert also modify the __str__ and __repr__ of newly created enumerations to display the module name instead of the enumeration name (https://bugs.python.org/msg325007):

--> socket.AF_UNIX ==>

--> print(socket.AF_UNIX) AddressFamily.AF_UNIX ==> socket.AF_UNIX

Thoughts?

-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GMOJRM4G... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

Chris Angelico

8:41 a.m.

On Thu, Mar 26, 2020 at 10:38 AM Ivan Pozdeev via Python-Dev wrote:

...

I'm skeptical about anything that hides an object's "true nature": this is a major landmine in diagnostics because it lies to you about what you are looking at and where to look for its implementation.

E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

But if it's described as socket.AF_UNIX then you know where to go looking for it, and type() can tell you the type if you actually need it. +1 for changing the socket enums; -1 for changing enums in general. ChrisA

Ivan Pozdeev

10:05 a.m.

On 26.03.2020 2:41, Chris Angelico wrote:

...

On Thu, Mar 26, 2020 at 10:38 AM Ivan Pozdeev via Python-Dev wrote:

...
I'm skeptical about anything that hides an object's "true nature": this is a major landmine in diagnostics because it lies to you about what you are looking at and where to look for its implementation.

E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

But if it's described as socket.AF_UNIX then you know where to go looking for it,

No, I don't. https://docs.python.org/3/library/functions.html#repr sets the standard -- thus the expectation -- for user-defined types: "...otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object" -- which this suggestion would be breaking. By this standard, "socket.AF_UNIX" looks like something defined directly at module level. So I'll be looking for a definition at module level -- and not find it. At which point, I'd suspect shenanigans like a dynamic creation or assignment. Which can be absolutely anywhere! And I'll be looking for the wrong thing still!

...

and type() can tell you the type if you actually needit.

That's an additional, unnecessary, nonstandard step that I somehow need to know to make because this is not standard practice, as per above. I do not expect such treachery, I expect repr() to give me truthful and accurate information! And even once I begin expecting it and additionally checking type() of everything that I examine from that point on (since I'll know that Python has crossed the line, I cannot trust repr() of anything any longer), that is lots of extra work, totally unwelcome and uncalled for! All for a tiny benefit, in one niche case.

...

+1 for changing the socket enums; -1 for changing enums in general.

ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O4XYZC7F... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

Chris Angelico

10:10 a.m.

On Thu, Mar 26, 2020 at 12:08 PM Ivan Pozdeev via Python-Dev wrote:

...

On 26.03.2020 2:41, Chris Angelico wrote:

...
On Thu, Mar 26, 2020 at 10:38 AM Ivan Pozdeev via Python-Dev wrote:

...
I'm skeptical about anything that hides an object's "true nature": this is a major landmine in diagnostics because it lies to you about what you are looking at and where to look for its implementation.

E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

But if it's described as socket.AF_UNIX then you know where to go looking for it,

No, I don't. https://docs.python.org/3/library/functions.html#repr sets the standard -- thus the expectation -- for user-defined types: "...otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object" -- which this suggestion would be breaking.

By this standard, "socket.AF_UNIX" looks like something defined directly at module level. So I'll be looking for a definition at module level -- and not find it. At which point, I'd suspect shenanigans like a dynamic creation or assignment. Which can be absolutely anywhere! And I'll be looking for the wrong thing still!

"""For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval()..."""

...

...
...
socket.AF_UNIX

If the repr were "socket.AF_UNIX", then that would indeed be a string that would yield an object with the same value. The current repr isn't wrong by this definition, but nor is the proposed change. Is it defined at module level? I couldn't care less; all that matters is that it is *accessible* at module level. I don't have to say "socket.AddressFamily.AF_UNIX". ChrisA

Ivan Pozdeev

10:53 a.m.

On 26.03.2020 4:10, Chris Angelico wrote:

...

On Thu, Mar 26, 2020 at 12:08 PM Ivan Pozdeev via Python-Dev wrote:

...
On 26.03.2020 2:41, Chris Angelico wrote:

...
On Thu, Mar 26, 2020 at 10:38 AM Ivan Pozdeev via Python-Dev wrote:

...
I'm skeptical about anything that hides an object's "true nature": this is a major landmine in diagnostics because it lies to you about what you are looking at and where to look for its implementation.

E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

But if it's described as socket.AF_UNIX then you know where to go looking for it, No, I don't. https://docs.python.org/3/library/functions.html#repr sets the standard -- thus the expectation -- for user-defined types: "...otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object" -- which this suggestion would be breaking.

By this standard, "socket.AF_UNIX" looks like something defined directly at module level. So I'll be looking for a definition at module level -- and not find it. At which point, I'd suspect shenanigans like a dynamic creation or assignment. Which can be absolutely anywhere! And I'll be looking for the wrong thing still!

"""For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval()...""" That clause is for the case without angle brackets. Then repr() must be exactly "socket.AF_INET", full stop. I.e. it will lose the integer value display.

...
...
...
socket.AF_UNIX

If the repr were "socket.AF_UNIX", then that would indeed be a string that would yield an object with the same value. The current repr isn't wrong by this definition, but nor is the proposed change. Is it defined at module level? I couldn't care less; all that matters is that it is *accessible* at module level. Well, I do care a lot. For diagnostics, this is a critical distinction. An object is defined in one, fixed, place (even if you overwrite module contents, it retains lexical scope) and can be made, dynamically, accessible in any number of completely unrelated places.

A diagnostic is always done by the same algorithm: 1) Identify the exact place in code where the problem manifests itself 2) Examine the state of the program at that moment to find out which if the values are wrong 3) Track the wrong value back to its origin Seeing an accurate repr() is required for step 2) -- to immediately detect if it's wrong or not. If a repr() points to a reference rather than definition, I don't really know anything about the object that I'm looking at: that reference could as well be wrong, or could have been wrong at any moment in the past. I.e. such a repr() has negative informativity. Finding the definition, as opposed to merely a reference, is required for step 3). To identify which code could have adversely affected the object's value, I must find both its implementation (for potential internal offenders) and references to it and objects of this type in general (for potential external offenders).

...

I don't have to say "socket.AddressFamily.AF_UNIX".

ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4Q7SVP6Z... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

Ethan Furman

27 Mar 27 Mar

1:24 a.m.

On 03/25/2020 06:53 PM, Ivan Pozdeev via Python-Dev wrote:

...

A diagnostic is always done by the same algorithm:

1) Identify the exact place in code where the problem manifests itself 2) Examine the state of the program at that moment to find out which if the values are wrong 3) Track the wrong value back to its origin

Seeing an accurate repr() is required for step 2) -- to immediately detect if it's wrong or not.

Before Enum, the repr for socket.AF_UNIX was: 1 Not very useful for debugging.

...

If a repr() points to a reference rather than definition, I don't really know anything about the object that I'm looking at: that reference could as well be wrong, or could have been wrong at any moment in the past. I.e. such a repr() has negative informativity.

Whether it's or it has light-years more information than it used to (okay, maybe only light-minutes ;) )

...

Finding the definition, as opposed to merely a reference, is required for step 3).

Before Enum, if you had tried to find AF_UNIX and did a full Lib/*.py search you would have found some comments and a couple lines of code -- never a definition. And that's assuming you made the leap from 1 to AF_UNIX. You should add searching for from ... import * to your debugging list, because then you'll find: from _socket import *

...

To identify which code could have adversely affected the object's value, I must find both its implementation (for potential internal offenders) and references to it and objects of this type in general (for potential external offenders).

Finding the implementation might be a bit trickier with `socket.AF_UNIX', however: $ grep AddressFamily.AF_UNIX *.py ...(crickets)... $ grep socket.AF_UNIX *.py smtplib.py: self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) socketserver.py: address_family = socket.AF_UNIX socketserver.py: address_family = socket.AF_UNIX webbrowser.py: s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) Considering that stdlib Enums are almost all globally unique, and Enums created with _convert *are* globally unique, I think seeing the module instead of the class name is more helpful. Speaking of which, adding help() to your debugging list would also be, um, helpful: --> help(socket.AF_UNIX) Help on AddressFamily in module socket object: class AddressFamily(enum.IntEnum) | AddressFamily(value, names=None, *, module=None, qualname=None, type=None, start=1) | | An enumeration. | | ... As a reminder, the change under discussion only affects enums created from globally unique constants using the Enum._convert helper utility. -- ~Ethan~

Leandro Müller

5:55 a.m.

New subject: sys.trace without alert or remove trace by C

Hello. I need to remove the trace on thread, but I tried to run sys.settrace(None) and return alert on console. I didn't find any soluction to remove by C, example PyEval_SetTrace(Py_None, obj), but it not works. Are there any soluction to remove without alert or by C? C is better for me. Thanks. Att. Leandro Müller

Victor Stinner

7:08 a.m.

New subject: sys.trace without alert or remove trace by C

Hi, I understand that you are looking for PyEval_SetTrace(NULL, NULL) to unregister a trace function set previously. Maybe the documentation should be enhanced to explain that. Do you want to propose a PR to enhance the doc? https://docs.python.org/dev/c-api/init.html?highlight=pyeval_settrace#c.PyEv... Victor Le jeu. 26 mars 2020 à 22:47, Leandro Müller a écrit :

...

Hello. I need to remove the trace on thread, but I tried to run sys.settrace(None) and return alert on console. I didn't find any soluction to remove by C, example PyEval_SetTrace(Py_None, obj), but it not works.

Are there any soluction to remove without alert or by C? C is better for me.

Thanks.

Att.

Leandro Müller

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HJNC3UOX... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

Leandro Müller

7:29 p.m.

New subject: sys.trace without alert or remove trace by C

Hello Victor. Thank you. The tips help me a lot. I think the tip needs to stay inside on document to reduce time to seach soluction. Thanks. Att. Leandro Müller ________________________________ De: Victor Stinner Enviado: Thursday, March 26, 2020 7:08:42 PM Para: Leandro Müller Cc: python-dev@python.org Assunto: Re: [Python-Dev] sys.trace without alert or remove trace by C Hi, I understand that you are looking for PyEval_SetTrace(NULL, NULL) to unregister a trace function set previously. Maybe the documentation should be enhanced to explain that. Do you want to propose a PR to enhance the doc? https://docs.python.org/dev/c-api/init.html?highlight=pyeval_settrace#c.PyEv... Victor Le jeu. 26 mars 2020 à 22:47, Leandro Müller a écrit :

...

Hello. I need to remove the trace on thread, but I tried to run sys.settrace(None) and return alert on console. I didn't find any soluction to remove by C, example PyEval_SetTrace(Py_None, obj), but it not works.

Are there any soluction to remove without alert or by C? C is better for me.

Thanks.

Att.

Leandro Müller

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HJNC3UOX... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

Ivan Pozdeev

28 Mar 28 Mar

9:21 a.m.

On 26.03.2020 19:24, Ethan Furman wrote:

...

On 03/25/2020 06:53 PM, Ivan Pozdeev via Python-Dev wrote:

...
A diagnostic is always done by the same algorithm:

1) Identify the exact place in code where the problem manifests itself 2) Examine the state of the program at that moment to find out which if the values are wrong 3) Track the wrong value back to its origin

Seeing an accurate repr() is required for step 2) -- to immediately detect if it's wrong or not.

Before Enum, the repr for socket.AF_UNIX was:

1

Not very useful for debugging.

On the contrary, it's perfect. I know everything I need: it's an integer constant, end of story. When examining some live object, I would probably see that "1" and won't even need to look anything up to know what I'm dealing with.

...

...
If a repr() points to a reference rather than definition, I don't really know anything about the object that I'm looking at: that reference could as well be wrong, or could have been wrong at any moment in the past. I.e. such a repr() has negative informativity.

Whether it's

or

it has light-years more information than it used to (okay, maybe only light-minutes ;) )

More information is not better if that information is harmful rather than helpful.

...

...
Finding the definition, as opposed to merely a reference, is required for step 3).

Before Enum, if you had tried to find AF_UNIX and did a full Lib/*.py search you would have found some comments and a couple lines of code -- never a definition. And that's assuming you made the leap from 1 to AF_UNIX.

You should add searching for

from ... import *

to your debugging list, because then you'll find:

from _socket import *

...
To identify which code could have adversely affected the object's value, I must find both its implementation (for potential internal offenders) and references to it and objects of this type in general (for potential external offenders).

Finding the implementation might be a bit trickier with `socket.AF_UNIX', however:

$ grep AddressFamily.AF_UNIX *.py ...(crickets)...

I would rather search for "AddressFamily" (case-sensitive), and in the entire codebase, not just "*.py". Long story short, this is a type name and looks like a very distinctive one. So that search is very likely going to readily find me anything directly related to that type, including its definition, with negligible noise -- however and wherever it's done. I hope this serves as yet another demonstration why knowing the exact type is so crucial.

...

$ grep socket.AF_UNIX *.py smtplib.py: self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) socketserver.py: address_family = socket.AF_UNIX socketserver.py: address_family = socket.AF_UNIX webbrowser.py: s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

Considering that stdlib Enums are almost all globally unique, and Enums created with _convert *are* globally unique, I think seeing the module instead of the class name is more helpful. Speaking of which, adding help() to your debugging list would also be, um, helpful:

--> help(socket.AF_UNIX) Help on AddressFamily in module socket object:

class AddressFamily(enum.IntEnum) | AddressFamily(value, names=None, *, module=None, qualname=None, type=None, start=1) | | An enumeration. | | ...

Examining repr() of a large number of objects (think dozens to hundreds) vs help() of each of them? I think we have a winner... And help() doesn't even show the value! You can't be serious suggesting me to do all this extra work (i.e. single-handedly increase diagnostic's labor intensity about 5x -- while problem diagnostic is a key activity during both development and ongoing use) just because someone wishes to playfully ignore an established standard specifically invented to make it unnecessary. Pardon me if I misunderstood you, but I also find the offhand remarks: "adding this and that to your debugging list would also be, um, helpful" -- highly condescending and insulting and showing that you probably don't do much problem diagnostic and don't care at all about anyone who does -- i.e. about software developers and maintainers, your core audience. All the while yourself being a Python core developer (if your StackOverflow profile is to be believed) -- a spokesperson for the dev team and a shining example of what every Python developer should strive to be. If this is any representation of the prevalent attitude in the core team now, I'm probably wasting my time arguing here as well as supporting Python in general since our values clearly don't match anymore.

...

As a reminder, the change under discussion only affects enums created from globally unique constants using the Enum._convert helper utility.

I know. But it sets a precedent of screwing users over with poorly thought cosmetic changes done without thinking of them. It becoming a pattern is the absolutely last thing that I want to see in Python since it being easy and intuitive to examine and manipulate live objects is one of its killer features that makes it the #1 middleware language among other things.

...

-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C55F5N4J... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

-- Regards, Ivan

Kyle Stanley

10:43 a.m.

Ivan Pozdeev wrote:

...

More information is not better if that information is harmful rather than helpful.

While that argument does apply in some cases, I'd have to very much disagree that "" is harmful in comparison to just "1"; it clearly shows the value on the right side of the colon. As for the source location for "socket.AF_UNIX" not being clear, that may be the case the first time you encounter it or haven't seen it in a while, but it's clear from a quick search of the help page. IMO, the *full* location shouldn't have to appear every time you want to see the repr; it would add unneeded noise. If you see the repr and it's not clear, consult the help page and/or docs for more information. The repr should be succinct.

...

And help() doesn't even show the value!

...

Pardon me if I misunderstood you, but I also find the offhand remarks: "adding this and that to your debugging list would also be, um, helpful" -- highly condescending and insulting and showing that you

Sorry, but this is simply untrue. If you scroll down on the help page, or search via "/" for "AF_UNIX", you'll encounter the following: | AF_UNIX = (which is likely what Ethan was referring to) If it wasn't clear, when you use "help(socket.AF_UNIX)", it brings up the AddressFamily help page (as indicated in the title), not a dedicated page for "socket.AF_UNIX". You can also find the information present in the "help(socket)" page if you search for "AF_UNIX". probably don't do much problem diagnostic and don't care at all about

...

anyone who does -- i.e. about software developers and maintainers, your core audience.

From my perspective, Ethan pointed out "help()" because it's very frequently neglected, even by experienced devs; not at all to be condescending towards you or insulting of your experience. However, explicitly claiming that "you probably don't do much problem diagnostic and don't care at all about anyone who does" is *directly* insulting and doesn't add anything constructive to the discussion. Insulting the other party, especially when they're responding to you voluntarily and not paid to do so, makes them far less likely to consider your side or even respond at all. Rather than the extreme of "either you agree with me or don't have experience with this", is it not possible that you simply disagree and have different preferences in what makes more for more optimal debugging information?

...

All the while yourself being a Python core developer (if your StackOverflow profile is to be believed) -- a spokesperson for the dev team and a shining example of what every Python developer should strive to be.

I'm not sure what the purpose of the "if your StackOverflow profile is to be believed" part is, but the list of Python core developers can be found here: https://devguide.python.org/developers/ (where "Ethan Furman" is clearly listed).

...

If this is any representation of the prevalent attitude in the core team now, I'm probably wasting my time arguing here as well as supporting Python in general since our values clearly don't match anymore.

I'm sorry that you feel that way. But if you considered the discussion to be a waste of time, then I don't think python-dev (or similar OSS dev communities) will be a very productive experience for you. Also, just to be clear: the opinion of one or a few core devs does not represent the opinions or values of the entire core team. The core team is made up of ~100 current individuals with differing opinions, areas of expertise, and perspectives. On Fri, Mar 27, 2020 at 8:27 PM Ivan Pozdeev via Python-Dev < python-dev@python.org> wrote:

...

...
On 03/25/2020 06:53 PM, Ivan Pozdeev via Python-Dev wrote:

...
A diagnostic is always done by the same algorithm:

1) Identify the exact place in code where the problem manifests itself 2) Examine the state of the program at that moment to find out which if

On 26.03.2020 19:24, Ethan Furman wrote: the values are wrong

...
...
3) Track the wrong value back to its origin

Seeing an accurate repr() is required for step 2) -- to immediately detect if it's wrong or not.

Before Enum, the repr for socket.AF_UNIX was:

1

Not very useful for debugging.

On the contrary, it's perfect. I know everything I need: it's an integer constant, end of story.

When examining some live object, I would probably see that "1" and won't even need to look anything up to know what I'm dealing with.

...
...
If a repr() points to a reference rather than definition, I don't

...
...
reference could as well be wrong, or could have been wrong at any moment in the past. I.e. such a repr() has negative informativity.

Whether it's

or

it has light-years more information than it used to (okay, maybe only

really know anything about the object that I'm looking at: that light-minutes ;) )

...
More information is not better if that information is harmful rather than helpful.

...
...
Finding the definition, as opposed to merely a reference, is required for step 3).

Before Enum, if you had tried to find AF_UNIX and did a full Lib/*.py search you would have found some comments and a couple lines of code -- never a definition. And that's assuming you made the leap from 1 to AF_UNIX.

You should add searching for

from ... import *

to your debugging list, because then you'll find:

from _socket import *

...
To identify which code could have adversely affected the object's value, I must find both its implementation (for potential internal offenders) and references to it and objects of this type in general (for potential external offenders).

Finding the implementation might be a bit trickier with `socket.AF_UNIX', however:

$ grep AddressFamily.AF_UNIX *.py ...(crickets)...

I would rather search for "AddressFamily" (case-sensitive), and in the entire codebase, not just "*.py". Long story short, this is a type name and looks like a very distinctive one. So that search is very likely going to readily find me anything directly related to that type, including its definition, with negligible noise -- however and wherever it's done.

I hope this serves as yet another demonstration why knowing the exact type is so crucial.

...
$ grep socket.AF_UNIX *.py smtplib.py: self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) socketserver.py: address_family = socket.AF_UNIX socketserver.py: address_family = socket.AF_UNIX webbrowser.py: s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

Considering that stdlib Enums are almost all globally unique, and Enums created with _convert *are* globally unique, I think seeing the module instead of the class name is more helpful. Speaking of which, adding help() to your debugging list would also be, um, helpful:

--> help(socket.AF_UNIX) Help on AddressFamily in module socket object:

class AddressFamily(enum.IntEnum) | AddressFamily(value, names=None, *, module=None, qualname=None, type=None, start=1) | | An enumeration. | | ...

Examining repr() of a large number of objects (think dozens to hundreds) vs help() of each of them? I think we have a winner... And help() doesn't even show the value!

You can't be serious suggesting me to do all this extra work (i.e. single-handedly increase diagnostic's labor intensity about 5x -- while problem diagnostic is a key activity during both development and ongoing use) just because someone wishes to playfully ignore an established standard specifically invented to make it unnecessary.

Pardon me if I misunderstood you, but I also find the offhand remarks: "adding this and that to your debugging list would also be, um, helpful" -- highly condescending and insulting and showing that you probably don't do much problem diagnostic and don't care at all about anyone who does -- i.e. about software developers and maintainers, your core audience. All the while yourself being a Python core developer (if your StackOverflow profile is to be believed) -- a spokesperson for the dev team and a shining example of what every Python developer should strive to be. If this is any representation of the prevalent attitude in the core team now, I'm probably wasting my time arguing here as well as supporting Python in general since our values clearly don't match anymore.

...
As a reminder, the change under discussion only affects enums created

from globally unique constants using the Enum._convert helper utility.

...
I know. But it sets a precedent of screwing users over with poorly thought cosmetic changes done without thinking of them. It becoming a pattern is the absolutely last thing that I want to see in Python since it being easy and intuitive to examine and manipulate live objects is one of its killer features that makes it the #1 middleware language among other things.

...
-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C55F5N4J... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

-- Regards, Ivan _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T2RJBWPJ... Code of Conduct: http://python.org/psf/codeofconduct/

Ethan Furman

11:10 a.m.

On 03/27/2020 05:21 PM, Ivan Pozdeev via Python-Dev wrote:

...

On 26.03.2020 19:24, Ethan Furman wrote:

...
On 03/25/2020 06:53 PM, Ivan Pozdeev via Python-Dev wrote:

...

...
...
A diagnostic is always done by the same algorithm:

1) Identify the exact place in code where the problem manifests itself 2) Examine the state of the program at that moment to find out which if the values are wrong 3) Track the wrong value back to its origin

Seeing an accurate repr() is required for step 2) -- to immediately detect if it's wrong or not.

Before Enum, the repr for socket.AF_UNIX was:

1

Not very useful for debugging.

On the contrary, it's perfect. I know everything I need: it's an integer constant, end of story.

So if it was a 3 you'd still be happy? 1 vs 3 gives you no information as to what the value should be, while blahblah.AF_UNIX vs blahblah.AF_AX25 could point out that that is the wrong value for the function you found it in.

...

When examining some live object, I would probably see that "1" and won't even need to look anything up to know what I'm dealing with.

If all you care about is the value, then blahblah.AF_UNIX isn't going to hurt you because the repr() displays it: . Hopefully that's immediately recognizable as an Enum, and a `type()` or `isinstance()` will tell you if it's also an `int`. [...]

...

I would rather search for "AddressFamily" (case-sensitive), and in the entire codebase, not just "*.py".

Yeah, looks like I messed up that example -- I must have switched directories in between the two. The actual results for my simple grep: $ grep AddressFamily *.py socket.py: "has_dualstack_ipv6", "AddressFamily", "SocketKind"] socket.py: 'AddressFamily', socket.py: return _intenum_converter(super().family, AddressFamily) socket.py: addrlist.append((_intenum_converter(af, AddressFamily), So, more useful for finding the source, still not useful for finding other uses.

...

Long story short, this is a type name and looks like a very distinctive one.

Yup, it's an Enum.

...

So that search is very likely going to readily find me anything directly related to that type, including its definition, with negligible noise -- however and wherever it's done.

Well, there are less than two dozen lines using your search, but the definition location is non-obvious -- all you see is `AddressFamily` Still, it's something.

...

I hope this serves as yet another demonstration why knowing the exact type is so crucial.

Not so much in this case -- if it says , then hopefully it's not a far stretch to look in the socket module.

...

...
--> help(socket.AF_UNIX) Help on AddressFamily in module socket object:

class AddressFamily(enum.IntEnum) | AddressFamily(value, names=None, *, module=None, qualname=None, type=None, start=1) | | An enumeration. | | ...

Examining repr() of a large number of objects (think dozens to hundreds) vs help() of each of them? I think we have a winner...

Well, the repr in question, whether `socket.*` or `AddressFamily.*`, is still distinctive and still contains the value, the thing you are saying you want.

...

And help() doesn't even show the value!

I had snipped the output for brevity. Here's the whole thing: Help on AddressFamily in module socket object: class AddressFamily(enum.IntEnum) | AddressFamily(value, names=None, *, module=None, qualname=None, type=None, start=1) | | An enumeration. | | Method resolution order: | AddressFamily | enum.IntEnum | builtins.int | enum.Enum | builtins.object | | Data and other attributes defined here: | | AF_ALG = | | AF_APPLETALK = | | AF_ASH = | | AF_ATMPVC = | | AF_ATMSVC = | | AF_AX25 = | | AF_BLUETOOTH = | | AF_BRIDGE = | | AF_CAN = | | AF_ECONET = | | AF_INET = | | AF_INET6 = | | AF_IPX = | | AF_IRDA = | | AF_KEY = | | AF_LLC = | | AF_NETBEUI = | | AF_NETLINK = | | AF_NETROM = | | AF_PACKET = | | AF_PPPOX = | | AF_QIPCRTR = | | AF_RDS = | | AF_ROSE = | | AF_SECURITY = | | AF_SNA = | | AF_TIPC = | | AF_UNIX = | | AF_UNSPEC = | | AF_VSOCK = | | AF_WANPIPE = | | AF_X25 = | | ---------------------------------------------------------------------- | Data descriptors inherited from enum.Enum: | | name | The name of the Enum member. | | value | The value of the Enum member. | | ---------------------------------------------------------------------- | Readonly properties inherited from enum.EnumMeta: | | __members__ | Returns a mapping of member name->value. | | This mapping lists all enum members, including aliases. Note that this | is a read-only view of the internal mapping. You'll note that all the members, and their values, are listed. (They say `socket` instead of `AddressFamily` as I was testing the change.)

...

You can't be serious suggesting me to do all this extra work (i.e. single-handedly increase diagnostic's labor intensity about 5x -- while problem diagnostic is a key activity during both development and ongoing use) just because someone wishes to playfully ignore an established standard specifically invented to make it unnecessary.

module.function module.class module.attribute are all common access patterns. It should not be a far stretch to see and think to look in the socket module.

...

Pardon me if I misunderstood you, but I also find the offhand remarks: "adding this and that to your debugging list would also be, um, helpful" -- highly condescending and insulting and showing that you probably don't do much problem diagnostic and don't care at all about anyone who does -- i.e. about software developers and maintainers, your core audience.

I was trying to be humorous -- my apologies. In fact, a large portion of my time is spent debugging. I also recognize Enums when I see them, so they aren't big mysteries to me -- admittedly, I am probably more familiar with Enums and their repr()s than most, but hopefully once you know what they look like they're easy to spot.

...

All the while yourself being a Python core developer (if your StackOverflow profile is to be believed) -- a spokesperson for the dev team and a shining example of what every Python developer should strive to be.

Well, yes to my Stackoverflow profile being correct, no, I was not made a spokesperson (my opinions are not necessarily shared by any other core developer, nor by the PSF), and do you expect every developer of every open source project to be "shining examples"? Hopefully you were just frustrated, and not serious.

...

If this is any representation of the prevalent attitude in the core team now, I'm probably wasting my time arguing here as well as supporting Python in general since our values clearly don't match anymore.

Well, my attitude is "ask on python-dev to get wide exposure and feedback to proposed changes since bugs on bpo are not widely seen."

...

...
As a reminder, the change under discussion only affects enums created from globally unique constants using the Enum._convert helper utility.

I know. But it sets a precedent of screwing users over with poorly thought cosmetic changes done without thinking of them.

That's offensive. I started this thread precisely because I'm thinking of them. -- ~Ethan~

Chris Angelico

11:13 a.m.

On Sat, Mar 28, 2020 at 11:28 AM Ivan Pozdeev via Python-Dev wrote:

...

On 26.03.2020 19:24, Ethan Furman wrote:

...
Before Enum, the repr for socket.AF_UNIX was:

1

Not very useful for debugging.

On the contrary, it's perfect. I know everything I need: it's an integer constant, end of story.

When examining some live object, I would probably see that "1" and won't even need to look anything up to know what I'm dealing with.

This is something I see *very* frequently from expert and novice Python programmers: the belief/assumption that seeing the concrete data is the most important thing. As an extreme expert in socket programming, you probably DO interpret the integer 1 as "AF_UNIX", but quite honestly, I wouldn't recognize it as such. But a repr that says that it's socket.AF_UNIX is far far more helpful. Consider: Suppose you have a call to socket.socket(socket.SOCK_STREAM, socket.AF_INET, socket.IPPROTO_TCP) - only they're not literal values, they're coming from elsewhere. Pre-enum, all you know is that it's being passed the integers 1, 2, and 6, and how are you going to figure out what they mean? Thanks to the enum, you get to see them with their names (which may make the bug obvious). But either way, they are "socket.AF_INET" etc, they are not "socket.AddressFamily.AF_INET" - the class is a mere implementation detail. In actual fact, the concrete integer value is just an implementation detail, too. As long as passing socket.AF_INET results in an internet-address-family socket, it doesn't really matter what the actual integer is. For the sake of cross-language debugging, it's helpful to be able to see it, but most Python programmers aren't going to need it. ChrisA

Serhiy Storchaka

26 Mar 26 Mar

5:59 p.m.

26.03.20 01:35, Ivan Pozdeev via Python-Dev пише:

...

E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

AF_UNIX is and always was available from the socket module. It is not incidental. AddressFamily is an implementation detail, it is just a way to add fanny repr and other properties to this constant. In any case the information about AddressFamily and its members will be kept in the help output.

Ivan Pozdeev

6:35 p.m.

On 26.03.2020 11:59, Serhiy Storchaka wrote:

...

26.03.20 01:35, Ivan Pozdeev via Python-Dev пише:

...
E. g. in this case, AF_UNIX is a member of some entity called "AddressFamily" -- so I would search the code for "AddressFamily" to see what I'm looking at and what else I can do with it. The fact that it's also directly availabe from the `socket` module is incidental: it could be as easily made available from anywhere else -- not even a module.

AF_UNIX is and always was available from the socket module. It is not incidental. AddressFamily is an implementation detail, it is just a way to add fanny repr and other properties to this constant. In any case the information about AddressFamily and its members will be kept in the help output.

See my earlier replies to Chris Angelico why showing the implementation matters for repr() and how this misrepresentation breaks the documented repr() standard and debugging.

...

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NNBYZBV7... Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan

-- Regards, Ivan

Ethan Furman

9:13 a.m.

On 03/25/2020 12:54 PM, Ethan Furman wrote:

...

--> socket.AF_UNIX ==>

--> print(socket.AF_UNIX) AddressFamily.AF_UNIX ==> socket.AF_UNIX

An important point I should have included -- the definition of AddressFamily: IntEnum._convert_( 'AddressFamily', __name__, lambda C: C.isupper() and C.startswith('AF_') ) That last line is filtering the globals from `__name__`* to make the Enum -- in other words, the member names are already globally unique. Every usage of `Enum._convert` is like that. -- ~Ethan~ * `__name__` is the module name, which is looked up in `sys.modules`.

Serhiy Storchaka

28 Mar 28 Mar

8:20 p.m.

25.03.20 21:54, Ethan Furman пише:

...

Serhiy had the idea of having Enum._convert also modify the __str__ and __repr__ of newly created enumerations to display the module name instead of the enumeration name (https://bugs.python.org/msg325007):

--> socket.AF_UNIX ==>

--> print(socket.AF_UNIX) AddressFamily.AF_UNIX ==> socket.AF_UNIX

Thoughts?

Thank you for raising this question Ethan. I have several other ideas, maybe not directly related to this. 1. Representation of the value. For IntFlags, values are bit masks, so it is more convenient to represent them as hexidecimals, octals or binaries than decimals. It may be useful for regular enumerations too. It would be nice to have a ways to customize the default representation of the value. For example by setting __value_repr__ = hex. 2. In many case it is more convenient to have a repr in the form Color.RED or even just RED instead of . First, it is shorter and contains all information that you need in the context. It is importatnt if it is a part of the larger object. Second, the result is evaluable, so you can just copy the output and paste it in the Python code, without removing noise. It is especially handy in tests, when you get a collection of enums, or dataclasses with enum fields, or more complex data structure and compare it with expected result. You just write a test, add print() for the actual value, run test, and copy the output as the expected value. So in these cases I want the repr be what the str is now. I do not say that it should be the behavior by default, but maybe change the repr for some stdlib enums for which the literal value is not important?