Allow Enum members to refer to each other during execution of body
Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body). Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?) Best, Antony
On 07/08/2013 02:27 PM, Antony Lee wrote:
Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example
class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table
StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A
It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body).
Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?)
Part of the problem here would be maintaining the linkage when the temp enum object from _EnumDict was translated into an actual Enum member. One possible work around is to store the name of the member instead: class StateMachine(Enum): A = {} B = {1:'A'} then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense? -- ~Ethan~
then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense?
Having real strings be part of the enums data members is a pretty common thing, and working through and trying to identify the linkage-strings from normal-strings seems very magical to me. Is there some metaclass-magic way to intercept the usage of A, to instead put the enum instance there? Also, for this to be useful for your described use case, (state machines yay!) you'd probably want to be able to define back/circular references, which i think isn't currently possible. The obvious thing to do would be to somehow make the RHS of the assignments lazy, which would allow out-of-order and circular assignments with a very nice, unambigious: class StateMachine(Enum): "Useless ping-pong state machine" A = {1: B} B = {1: A} But short of using macros to do an AST transform, I don't know if such a thing is possible at all. -Haoyi On Tue, Jul 9, 2013 at 8:12 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 07/08/2013 02:27 PM, Antony Lee wrote:
Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example
class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table
StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A
It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body).
Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?)
Part of the problem here would be maintaining the linkage when the temp enum object from _EnumDict was translated into an actual Enum member.
One possible work around is to store the name of the member instead:
class StateMachine(Enum): A = {} B = {1:'A'}
then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense?
-- ~Ethan~ ______________________________**_________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
On 07/08/2013 06:03 PM, Haoyi Li wrote: On 07/08/2013 Ethan Furman wrote:
then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense?
Having real strings be part of the enums data members is a pretty common thing, and working through and trying to identify the linkage-strings from normal-strings seems very magical to me. Is there some metaclass-magic way to intercept the usage of A, to instead put the enum instance there?
The post-processing routines would be specific to this enumeration, so they should know which strings are enum references and which aren't.
Also, for this to be useful for your described use case, (state machines yay!) you'd probably want to be able to define back/circular references, which i think isn't currently possible. The obvious thing to do would be to somehow make the RHS of the assignments lazy, which would allow out-of-order and circular assignments with a very nice, unambigious:
class StateMachine(Enum): "Useless ping-pong state machine" A = {1: B} B = {1: A}
But short of using macros to do an AST transform, I don't know if such a thing is possible at all.
Starting off with strings and post-processing would handle it nicely. -- ~Ethan~
Forward references are now implemented (https://github.com/anntzer/enum). They require an explicit declaration, à la class C(Enum, declaration=...): B = ... A = {1: B} B = {1: A} I had implemented a version where the initial declaration wasn't needed, but as mentioned in previous enum-related threads this can create many problems. For example, consider class C(Enum): A = {1: B}; B = {1: A} @property def also_value(self): return self.value how is Python supposed to know that when it tries to resolve "B" in the class dict, it must create a new member, but when it tries to resolve "property" in the class dict and doesn't find it, it must look in the enclosing scope? You can decide that a name lookup creates a new member if the name isn't defined in the enclosing scope either (I implemented this using sys._getframe in a previous commit of my fork) but this leads to other (somewhat contrieved) problems: x = 1 class C(Enum): y = x # <- what is this supposed to mean? x = 2 Note that even AST macros don't (fully) this issue because you can't really even know the list of all names that are defined in the class body: x = 1 def inject(**kwargs): for k, v in kwargs.items(): sys._getframe(1).f_locals[k] = v # interestingly using dict.update does not trigger the use-defined __setitem__ class C(Enum): y=x # <- ??? inject_value(x=2) Antony On Monday, July 8, 2013 6:03:27 PM UTC-7, Haoyi Li wrote:
then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense?
Having real strings be part of the enums data members is a pretty common thing, and working through and trying to identify the linkage-strings from normal-strings seems very magical to me. Is there some metaclass-magic way to intercept the usage of A, to instead put the enum instance there?
Also, for this to be useful for your described use case, (state machines yay!) you'd probably want to be able to define back/circular references, which i think isn't currently possible. The obvious thing to do would be to somehow make the RHS of the assignments lazy, which would allow out-of-order and circular assignments with a very nice, unambigious:
class StateMachine(Enum): "Useless ping-pong state machine" A = {1: B} B = {1: A}
But short of using macros to do an AST transform, I don't know if such a thing is possible at all.
-Haoyi
On Tue, Jul 9, 2013 at 8:12 AM, Ethan Furman <et...@stoneleaf.us<javascript:>
wrote:
On 07/08/2013 02:27 PM, Antony Lee wrote:
Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example
class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table
StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A
It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body).
Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?)
Part of the problem here would be maintaining the linkage when the temp enum object from _EnumDict was translated into an actual Enum member.
One possible work around is to store the name of the member instead:
class StateMachine(Enum): A = {} B = {1:'A'}
then the other methods can either dereference the name with an __getitem__ look-up, or the class can be post-processed with a decorator to change the strings back to actual members... hmmm, maybe a post_process hook in the metaclass would make sense?
-- ~Ethan~ ______________________________**_________________ Python-ideas mailing list Python...@python.org <javascript:> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
On 07/10/2013 03:47 PM, Antony Lee wrote:
Forward references are now implemented (https://github.com/anntzer/enum). They require an explicit declaration, à la
Do they work with a custom __new__ ? __init__ ? -- ~Ethan~
In the current version, they work with a custom __init__ (though of course, as long as the actual arguments that need to be passed to __init__ are provided, the pre-declared members are just "empty"). They do not work with a custom __new__ (not sure how I could make this work, given that at declaration time an "empty" member needs to be created but we don't know what arguments we need to pass to __new__...). As a side effect, however, the whole patch adds a new requirement: custom __new__s must be defined before the members themselves; otherwise they won't be called, for the same reason as above: if I don't know what __new__ is, I can't call it... Antony On Wednesday, July 10, 2013 6:52:24 PM UTC-7, stoneleaf wrote:
On 07/10/2013 03:47 PM, Antony Lee wrote:
Forward references are now implemented (https://github.com/anntzer/enum).
They require an explicit declaration, à la
Do they work with a custom __new__ ? __init__ ?
-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org <javascript:> http://mail.python.org/mailman/listinfo/python-ideas
On 07/11/2013 02:07 PM, Antony Lee wrote:
On Wednesday, July 10, 2013 6:52:24 PM UTC-7, stoneleaf wrote:
On 07/10/2013 03:47 PM, Antony Lee wrote:
Forward references are now implemented (https://github.com/anntzer/enum <https://github.com/anntzer/enum>).
Do they work with a custom __new__ ? __init__ ?
In the current version, they work with a custom __init__ (though of course, as long as the actual arguments that need to be passed to __init__ are provided, the pre-declared members are just "empty"). They do not work with a custom __new__ (not sure how I could make this work, given that at declaration time an "empty" member needs to be created but we don't know what arguments we need to pass to __new__...). As a side effect, however, the whole patch adds a new requirement: custom __new__s must be defined before the members themselves; otherwise they won't be called, for the same reason as above: if I don't know what __new__ is, I can't call it...
Hmm. Well, at this point I can offer kudos for getting it this far, but that's about it. The use-case this addresses seems fairly rare, and is definitely not a typical enumeration, and can be solved fairly easily with some extra post-processing code on a per-enumeration basis. -- ~Ethan~
"Part of the problem here would be maintaining the linkage when the temp enum object from _EnumDict was translated into an actual Enum member." I implemented the required behavior here: https://github.com/anntzer/enum Instead of creating a new enum object from the temp object stored in _EnumDict, I directly change the class of the temp object to its actual value once that class is built, thus keeping references correct (see test_backward_reference). However, this behavior breaks down if the Enum class also inherits from a type with a different layout (e.g., int), because I can't change the class of such objects. In fact, even class A(int): pass class B(int): pass A().__class__ = B fails (which is puzzling to me... I understand that you can't transform an instance of an int-subclass into an instance of an str-subclass), but here both classes should have the same layout... On the other hand IntEnums shouldn't need that kind of behavior anyways, so I just kept the old implementation for them (for any class for which instances can't be instantiated by object.__new__(cls), in fact). I haven't worked on forward-references but this should be not too hard to implement either: just add a __missing__ to _EnumDict (cf. the discussion on implicit enums) that creates temporary placeholder members on the fly. When these members are actually defined, initialize them. When class body finishes to execute, check that all placeholders have been initialized, throwing an error otherwise. Antony 2013/7/8 Antony Lee <antony.lee@berkeley.edu>
Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example
class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table
StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A
It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body).
Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?)
Best,
Antony
participants (4)
-
Antony Lee -
Antony Lee -
Ethan Furman -
Haoyi Li