I just received this bug report in psutil: https://code.google.com/p/psutil/issues/detail?id=408 Long story short, in psutil I have defined a constant type: http://code.activestate.com/recipes/577984-named-constant-type/ https://code.google.com/p/psutil/source/browse/psutil/_common.py#31 ...behaving along these lines:
FOO = constant(0, 'FOO') FOO 0 str(FOO) 'FOO'
It's a nasty problem and I'm still not sure how to solve it in psutil but this got me thinking about PEP 435 (Adding an Enum type to the Python standard library) since the two things are kind of related. I haven't properly gone through the PEP yet (I will) but I notice it doesn't talk about serialization. Has it been considered? Regards, - Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/
On 07/18/2013 11:13 AM, Giampaolo Rodola' wrote:
I just received this bug report in psutil: https://code.google.com/p/psutil/issues/detail?id=408
Long story short, in psutil I have defined a constant type: http://code.activestate.com/recipes/577984-named-constant-type/ https://code.google.com/p/psutil/source/browse/psutil/_common.py#31 ...behaving along these lines:
FOO = constant(0, 'FOO') FOO 0 str(FOO) 'FOO'
It's a nasty problem and I'm still not sure how to solve it in psutil but this got me thinking about PEP 435 (Adding an Enum type to the Python standard library) since the two things are kind of related. I haven't properly gone through the PEP yet (I will) but I notice it doesn't talk about serialization. Has it been considered?
See http://bugs.python.org/issue18264. The basic issue is that json only knows how to serialize built-in types. As soon as we build on that, json barfs. One way around that is to write your own json handler that knows about your custom types. For the 3.4 stdlib there are two proposals on the table: 1) for IntEnum and FloatEnum cast the member to int or float, then procede; 2) for any Enum, extract the value and proceed. -- ~Ethan~
On Thu, Jul 18, 2013 at 8:24 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
Oh, OK. Glad to see this is being tracked.
The basic issue is that json only knows how to serialize built-in types. As soon as we build on that, json barfs.
One way around that is to write your own json handler that knows about your custom types.
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
What about third party serialization libs though? --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/
On 07/18/2013 04:54 PM, Giampaolo Rodola' wrote:
On Thu, Jul 18, 2013 at 8:24 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
Oh, OK. Glad to see this is being tracked.
The basic issue is that json only knows how to serialize built-in types. As soon as we build on that, json barfs.
One way around that is to write your own json handler that knows about your custom types.
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
What about third party serialization libs though?
They are in the same boat, and have the same options. Short of defining a new protocol, I don't think there's much we can do for other serialization libs, besides making it easy for them to do the same thing we are. -- ~Ethan~
On Jul 18, 2013, at 05:01 PM, Ethan Furman wrote:
What about third party serialization libs though?
They are in the same boat, and have the same options.
Third party serializations (and built-in JSON serialization) already has to be taught about several common built-in data types, e.g. datetimes and timedeltas. If it weren't for the rather icky json module API for extensions, it's not that hard. -Barry
Could we start using @singledispatch? --Guido van Rossum (sent from Android phone) On Jul 19, 2013 8:27 AM, "Barry Warsaw" <barry@python.org> wrote:
On Jul 18, 2013, at 05:01 PM, Ethan Furman wrote:
What about third party serialization libs though?
They are in the same boat, and have the same options.
Third party serializations (and built-in JSON serialization) already has to be taught about several common built-in data types, e.g. datetimes and timedeltas. If it weren't for the rather icky json module API for extensions, it's not that hard.
-Barry
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On 07/19/2013 07:01 AM, Barry Warsaw wrote:
On Jul 19, 2013, at 08:30 AM, Guido van Rossum wrote:
Could we start using @singledispatch?
Yeah, that's a really interesting idea.
Possibly a (very) stupid question, but how would @singledispatch work with the _json accelerator module? -- ~Ethan~
On 19 July 2013 04:24, Ethan Furman <ethan@stoneleaf.us> wrote:
On 07/18/2013 11:13 AM, Giampaolo Rodola' wrote:
I just received this bug report in psutil: https://code.google.com/p/psutil/issues/detail?id=408
Long story short, in psutil I have defined a constant type: http://code.activestate.com/recipes/577984-named-constant-type/ https://code.google.com/p/psutil/source/browse/psutil/_common.py#31 ...behaving along these lines:
FOO = constant(0, 'FOO') FOO
0
str(FOO)
'FOO'
It's a nasty problem and I'm still not sure how to solve it in psutil but this got me thinking about PEP 435 (Adding an Enum type to the Python standard library) since the two things are kind of related. I haven't properly gone through the PEP yet (I will) but I notice it doesn't talk about serialization. Has it been considered?
See http://bugs.python.org/issue18264.
The basic issue is that json only knows how to serialize built-in types. As soon as we build on that, json barfs.
One way around that is to write your own json handler that knows about your custom types.
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
3) Only change repr (not str) in IntEnum I know Guido doesn't like it, but I still think the backwards compatibility risk is too high to use them to replace constants in the standard library if they change the output of __str__. Debugging only needs repr, and we can make sure stdlib error messages use repr, too. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
So you want to make print(<intenum>) output '42' instead of 'THE_ANSWER'? I am strongly against that change that. On Thu, Jul 18, 2013 at 10:11 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 19 July 2013 04:24, Ethan Furman <ethan@stoneleaf.us> wrote:
On 07/18/2013 11:13 AM, Giampaolo Rodola' wrote:
I just received this bug report in psutil: https://code.google.com/p/psutil/issues/detail?id=408
Long story short, in psutil I have defined a constant type: http://code.activestate.com/recipes/577984-named-constant-type/ https://code.google.com/p/psutil/source/browse/psutil/_common.py#31 ...behaving along these lines:
FOO = constant(0, 'FOO') FOO
0
str(FOO)
'FOO'
It's a nasty problem and I'm still not sure how to solve it in psutil but this got me thinking about PEP 435 (Adding an Enum type to the Python standard library) since the two things are kind of related. I haven't properly gone through the PEP yet (I will) but I notice it doesn't talk about serialization. Has it been considered?
See http://bugs.python.org/issue18264.
The basic issue is that json only knows how to serialize built-in types. As soon as we build on that, json barfs.
One way around that is to write your own json handler that knows about your custom types.
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
3) Only change repr (not str) in IntEnum
I know Guido doesn't like it, but I still think the backwards compatibility risk is too high to use them to replace constants in the standard library if they change the output of __str__. Debugging only needs repr, and we can make sure stdlib error messages use repr, too.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (python.org/~guido)
On 07/18/2013 10:11 PM, Nick Coghlan wrote:
On 19 July 2013 04:24, Ethan Furman <ethan@stoneleaf.us> wrote:
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
3) Only change repr (not str) in IntEnum
I know Guido doesn't like it, but I still think the backwards compatibility risk is too high to use them to replace constants in the standard library if they change the output of __str__. Debugging only needs repr, and we can make sure stdlib error messages use repr, too.
The issue with that solution is json uses repr, not str, for floats. So we would still have a problem. -- ~Ethan~
On 20 July 2013 02:07, Ethan Furman <ethan@stoneleaf.us> wrote:
On 07/18/2013 10:11 PM, Nick Coghlan wrote:
On 19 July 2013 04:24, Ethan Furman <ethan@stoneleaf.us> wrote:
For the 3.4 stdlib there are two proposals on the table:
1) for IntEnum and FloatEnum cast the member to int or float, then procede;
2) for any Enum, extract the value and proceed.
3) Only change repr (not str) in IntEnum
I know Guido doesn't like it, but I still think the backwards compatibility risk is too high to use them to replace constants in the standard library if they change the output of __str__. Debugging only needs repr, and we can make sure stdlib error messages use repr, too.
The issue with that solution is json uses repr, not str, for floats. So we would still have a problem.
Gah, I forgot about that (and I think it came up on the issue tracker, too). Perhaps we should include a "Converting existing constants to Enums" section in the enum docs, noting some of the backwards compatibility implications for serialisation when using IntEnum or similar subtypes? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (5)
-
Barry Warsaw -
Ethan Furman -
Giampaolo Rodola' -
Guido van Rossum -
Nick Coghlan