[Python-Dev] Proposed resolutions for open PEP 343 issues

Nick Coghlan ncoghlan at iinet.net.au
Sat Oct 22 15:58:48 CEST 2005

I'm still looking for more feedback on the issues raised in the last update of 
PEP 343. There hasn't been much direct feedback so far, but I've rephrased and 
suggested resolutions for the outstanding issues based on what feedback I have 
received, and my own thoughts over the last week of so.

For those simply skimming, my proposed issue resolutions are:

   1. Use the slot name "__context__" instead of "__with__"
   2. Reserve the builtin name "context" for future use as described below
   3a. Give generator-iterators a native context that invokes self.close()
   3b. Use "contextmanager" as a builtin decorator to get generator-contexts
   4. Special case the __context__ slot to avoid the need to decorate it

For those that disagree with the proposed resolutions above, or simply would 
like more details, here's the reasoning:

1. Should the slot be named "__with__" or "__context__"?

     Guido raised this as a side comment during the discussion of
   PJE's task variables pre-PEP, and it's a fair question.
     The closest analogous slot method ("__iter__") is named after the protocol
   it relates to, rather than the associated statement/expression keyword (that
   is, the method isn't called "__for__").
     The next closest analogous slot is one that doesn't actually exist yet -
   the proposed "boolean" protocol. This again uses the name of a protocol
   rather than the associated keyword (that is, the name "__bool__" was
   suggested rather than "__if__").
     At the moment, PEP 343 makes the opposite choice - it uses the keyword,
   rather than the protocol name (that is, it uses "__with__" instead of using
     That inconsistency would be a bad thing, in my opinion, and I propose that
   the slot should instead be named "__context__".

2. If the slot is called "__context__" what should a "context" builtin do?

     Again, considering existing slot names, a slot with a given name is
   generally invoked by the builtin type or function with the same name.
     This is true of the builtin types, and also true of iter, cmp and pow.
   getattr, setattr and delattr get in on the act as well.
     So, to be consistent, one would expect a "context" builtin to be able to
   be used such that "context(x)" invoked "x.__context__()".
     Such a method might special-case certain types, or have a two-argument
   form that accepted an "enter" function and an "exit" function, but using it
   to mark a generator that is to be used as a context manager (as currently
   suggested in PEP 343) would not be appropriate.
     I don't mind either way whether or not a "context" builtin is actually
   included for Python 2.5. However, even if it isn't included, the name should
   be reserved for that purpose (that is, we shouldn't be using it to name a
   generator decorator or a module).

3. How should generators behave when used as contexts?

     With PEP 342 accepted, generators pose a problem, because they have two
   possible uses as contexts. The first is for a generator that is intended to
   be used as an actual iterator. This case is a case of resource management -
   ensuring the close method is invoked on the generator-iterator when done
   with it (i.e., similar to the proposed native context for files).
     PEP 343 proposes a second use case for generators - to write custom
   context managers. In this case, the __enter__ method steps the generator
   once, and the __exit__ method finishes the job.
     I propose that we give generator-iterator objects resource management
   behaviour by default (i.e., __context__ and __enter__ methods that just
   "return self", and an __exit__ method that invokes "self.close()").
     The "contextmanager" builtin decorator from previous drafts of the PEP
   (called simply "context" in the current draft) can then be used to get the
   custom context manager behaviour.
     I previously thought giving generators a native context caused problems
   with getting silent failures when the "contextmanager" decorator was
   inadvertently omitted. This is still technically true - the "with" statement
   itself won't raise a TypeError because the generator is a legal context.
     However, with this bug, the context manager won't be getting entered *at
   all* (it gets closed without its next() method ever being called). Even the
   most cursory testing of the generator-context function should be able to
   tell whether the generator-context is being entered or not.
     The main alternative (having yet-another-decorator to give generators
   "auto-close" functionality) would be possible, but the additional builtin
   clutter would be getting to the point where it started to concern me. And
   given that "yield" inside "try/finally" is now always legal, I consider it
   reasonable that using a generator in a "with" statement would also always be
     Further, if type_new special cases __context__ as suggested below, then
   the context behaviour of generators used to define "__iter__" and
   "__context__" slots will always be appropriate.

4. Should the __context__ slot be special-cased in type_new?

     Currently, type_new special cases the "__new__" slot and automatically
   applies the staticmethod decorator when it finds a function occupying that
   slot in the class attribute dictionary.
     I propose that type_new also special case the situation where the
   "__context__" slot is occupied by a generator function, and automatically
   apply the "contextmanager" decorator.
     This looks much nicer when using a generator to write a __context__
   function, and also avoids the situation where the decorator is omitted, and
   the object becomes legal to use directly in with statements but doesn't
   actually do the right thing.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list