On 05/13/2013 11:32 AM, Guido van Rossum wrote:
But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an unpicklable enum. That's not a great experience. Pickling and unpickling errors are often remarkably hard to debug. (Especially the latter, so I have privately admonished Ethan to ensure that if the getframe hack doesn't work, the pickle failure should happen at pickling time, not at unpickle time.)
I can get pickle failure on members created using the functional syntax with no module set; I cannot get pickle failure on those same classes; I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests). If anybody has any insight on how to make that work, I'm all ears. -- ~Ethan~
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 05/13/2013 11:32 AM, Guido van Rossum wrote:
But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an unpicklable enum. That's not a great experience. Pickling and unpickling errors are often remarkably hard to debug. (Especially the latter, so I have privately admonished Ethan to ensure that if the getframe hack doesn't work, the pickle failure should happen at pickling time, not at unpickle time.)
I can get pickle failure on members created using the functional syntax with no module set;
That's the case I care most about.
I cannot get pickle failure on those same classes;
I suppose you mean "if you create the same enums using class syntax"? Sounds fine to me.
I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests).
Is the NEI base class picklable?
If anybody has any insight on how to make that work, I'm all ears.
I'm not 100% sure I know what "that" refers to here. -- --Guido van Rossum (python.org/~guido)
On 05/14/2013 01:58 PM, Guido van Rossum wrote:
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 05/13/2013 11:32 AM, Guido van Rossum wrote:
But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an unpicklable enum. That's not a great experience. Pickling and unpickling errors are often remarkably hard to debug. (Especially the latter, so I have privately admonished Ethan to ensure that if the getframe hack doesn't work, the pickle failure should happen at pickling time, not at unpickle time.)
I can get pickle failure on members created using the functional syntax with no module set;
That's the case I care most about.
Good, 'cause that one is handled. :)
I cannot get pickle failure on those same classes;
I suppose you mean "if you create the same enums using class syntax"? Sounds fine to me.
No. Example class: --> Example = Enum('Example', 'example ie eg') # no module name given, frame hack fails --> pickle(Example.ie) # blows up --# pickle(Example) # succeeds here, but unpickle will fail
I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests).
Is the NEI base class picklable?
No. If it is, then the derived enum is also picklable (at least the variation I have tested, which is when the NEI base class has __getnewargs__). I'm really hoping you'll say that can be a documentation issue. ;) -- ~Ethan~
On Tue, May 14, 2013 at 2:13 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 05/14/2013 01:58 PM, Guido van Rossum wrote:
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
I can get pickle failure on members created using the functional syntax with no module set;
That's the case I care most about.
Good, 'cause that one is handled. :)
Then we're good.
I cannot get pickle failure on those same classes;
I suppose you mean "if you create the same enums using class syntax"? Sounds fine to me.
No. Example class:
--> Example = Enum('Example', 'example ie eg') # no module name given, frame hack fails
--> pickle(Example.ie) # blows up
--# pickle(Example) # succeeds here, but unpickle will fail
Not great, but (a) few people pickle classes, and (b) there's probably something you can do to the metaclass to sabotage this. But it's fine to punt on this now.
I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests).
Is the NEI base class picklable?
No. If it is, then the derived enum is also picklable (at least the variation I have tested, which is when the NEI base class has __getnewargs__).
I'm really hoping you'll say that can be a documentation issue. ;)
Essentially the same response -- with enough hackery you can probably get this to do what you want, but I wouldn't hold up a release for it. For example you could file low-priority bugs for both issues in the hope that someone else figures it out. -- --Guido van Rossum (python.org/~guido)
On 15 May 2013 07:38, "Guido van Rossum" <guido@python.org> wrote:
On Tue, May 14, 2013 at 2:13 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 05/14/2013 01:58 PM, Guido van Rossum wrote:
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman <ethan@stoneleaf.us>
wrote:
I can get pickle failure on members created using the functional syntax with no module set;
That's the case I care most about.
Good, 'cause that one is handled. :)
Then we're good.
I cannot get pickle failure on those same classes;
I suppose you mean "if you create the same enums using class syntax"? Sounds fine to me.
No. Example class:
--> Example = Enum('Example', 'example ie eg') # no module name given, frame hack fails
--> pickle(Example.ie) # blows up
--# pickle(Example) # succeeds here, but unpickle will fail
Not great, but (a) few people pickle classes, and (b) there's probably something you can do to the metaclass to sabotage this. But it's fine to punt on this now.
It may be a bug in pickle - it sounds like it is sanity checking type(obj), but not checking for cases where obj itself is a class. Cheers, Nick.
I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests).
Is the NEI base class picklable?
No. If it is, then the derived enum is also picklable (at least the variation I have tested, which is when the NEI base class has __getnewargs__).
I'm really hoping you'll say that can be a documentation issue. ;)
Essentially the same response -- with enough hackery you can probably get this to do what you want, but I wouldn't hold up a release for it.
For example you could file low-priority bugs for both issues in the hope that someone else figures it out.
-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
On 05/14/2013 03:16 PM, Nick Coghlan wrote:
On 15 May 2013 07:38, "Guido van Rossum" <guido@python.org <mailto:guido@python.org>> wrote:
On Tue, May 14, 2013 at 2:13 PM, Ethan Furman <ethan@stoneleaf.us <mailto:ethan@stoneleaf.us>> wrote:
On 05/14/2013 01:58 PM, Guido van Rossum wrote:
On Tue, May 14, 2013 at 1:09 PM, Ethan Furman <ethan@stoneleaf.us <mailto:ethan@stoneleaf.us>> wrote:
I can get pickle failure on members created using the functional syntax with no module set;
That's the case I care most about.
Good, 'cause that one is handled. :)
Then we're good.
I cannot get pickle failure on those same classes;
I suppose you mean "if you create the same enums using class syntax"? Sounds fine to me.
No. Example class:
--> Example = Enum('Example', 'example ie eg') # no module name given, frame hack fails
--> pickle(Example.ie) # blows up
--# pickle(Example) # succeeds here, but unpickle will fail
Not great, but (a) few people pickle classes, and (b) there's probably something you can do to the metaclass to sabotage this. But it's fine to punt on this now.
It may be a bug in pickle - it sounds like it is sanity checking type(obj), but not checking for cases where obj itself is a class.
Well, it's definitely not calling the metaclass' __reduce__ as that's where I put the bomb (hmm, will I be visited by men in dark suits now?) so maybe that's a bug in pickle. At any rate, I figured it out -- give the class' __module__ a dummy name (I like 'uh uh' ;) and when pickle can't find that module it'll blow itself up. -- ~Ethan~
participants (3)
-
Ethan Furman -
Guido van Rossum -
Nick Coghlan