[Python-Dev] PEP 435: pickling enums created with the functional API

Eli Bendersky eliben at gmail.com
Tue May 7 15:34:50 CEST 2013


One of the contended issues with PEP 435 on which Guido pronounced was the
functional API, that allows created enumerations dynamically in a manner
similar to namedtuple:

  Color = Enum('Color', 'red blue green')

The biggest complaint reported against this API is interaction with pickle.
As promised, I want to discuss here how we're going to address this concern.

At this point, the pickle docs say that module-top-level classes can be
pickled. This obviously works for the normal Enum classes, but is a problem
with the functional API because the class is created dynamically and has no
__module__.

To solve this, the reference implementation is used the same approach as
namedtuple (*). In the metaclass's __new__ (this is an excerpt, the real
code has some safeguards):

  module_name = sys._getframe(1).f_globals['__name__']
  enum_class.__module__ = module_name

According to an earlier discussion, this is works on CPython, PyPy and
Jython, but not on IronPython. The alternative that works everywhere is to
define the Enum like this:

  Color = Enum('the_module.Color', 'red blue green')

The reference implementation supports this as well.

Some points for discussion:

1) We can say that using the functional API when pickling can happen is not
recommended, but maybe a better way would be to just explain the way things
are and let users decide?

2) namedtuple should also support the fully qualified name syntax. If this
is agreed upon, I can create an issue.

3) Antoine mentioned that work is being done in 3.4 to enable pickling of
nested classes (http://www.python.org/dev/peps/pep-3154/). If that gets
implemented, I don't see a reason why Enum and namedtuple can't be adjusted
to find the __qualname__ of the class they're internal to. Am I missing
something?

4) Using _getframe(N) here seems like an overkill to me. What we really
need is just the module in which the current execution currently is (i.e.
the metaclass's __new__ in our case). Would it make sense to add a new
function somewhere in the stdlib of 3.4 (in sys or inspect or ...) that
just provides the current module name? It seems that all Pythons should be
able to easily provide it, it's certainly a very small subset of the
functionality provided by walking the callframe stack. This function can
then be used for build fully qualified names for pickling of Enum and
namedtuple. Moreover, it can be general even more widely - dynamic class
building is quite common in Python code, and as Nick mentioned somewhere
earlier, the extra power of metaclasses in the recent 3.x's will probably
make it even more common.

Eli


(*) namedtuple uses an explicit function to build the resulting class, not
a metaclass (ther's no class syntax for namedtuple).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130507/7e26c729/attachment.html>


More information about the Python-Dev mailing list