
Based on Jason's comments regarding decimal.Context, and to explicitly cover the terminology agreed on during the documentation discussion back in July, I'm proposing a number of changes to PEP 343. I'll be updating the checked in PEP assuming there aren't any objections in the next week or so (and assuming I get CVS access sorted out ;). The idea of dropping __enter__/__exit__ and defining the with statement solely in terms of coroutines is *not* included in the suggested changes, but I added a new item under "Resolved Open Issues" to cover some of the reasons why. Cheers, Nick. 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) 2. Add the following to the subsequent explanation: The call to the __with__ method serves a similar purpose to the __iter__ method for iterables and iterators. An object such as threading.Lock may provide its own __enter__ and __exit__ methods, and simply return 'self' from its __with__ method. A more complex object such as decimal.Context may return a distinct context manager which takes care of setting and restoring the appropriate decimal context in the thread. 3. Update ContextWrapper in the "Generator Decorator" section to include: def __with__(self): return self 4. Add a paragraph to the end of the "Generator Decorator" section: By applying the @contextmanager decorator to a context's __with__ method, it is as easy to write a generator-based context manager for the context as it is to write a generator-based iterator for an iterable (see the decimal.Context example below). 5. Add three items under "Resolved Open Issues": 2. After this PEP was originally approved, a subsequent discussion on python-dev [4] settled on the term "context manager" for objects which provide __enter__ and __exit__ methods, and "context management protocol" for the protocol itself. With the addition of the __with__ method to the protocol, a natural extension is to call objects which provide only a __with__ method "contexts" (or "manageable contexts" in situations where the general term "context" would be ambiguous). The distinction between a context and a context manager is very similar to the distinction between an iterable and an iterator. 3. The originally approved version of this PEP did not include a __with__ method - the method was only added to the PEP after Jason Orendorff pointed out the difficulty of writing appropriate __enter__ and __exit__ methods for decimal.Context [5]. This approach allows a class to use the @contextmanager decorator to defines a native context manager using generator syntax. It also allows a class to use an existing independent context manager as its native context manager by applying the independent context manager to 'self' in its __with__ method. It even allows a class written in C to use a coroutine based context manager written in Python. The __with__ method parallels the __iter__ method which forms part of the iterator protocol. 4. The suggestion was made by Jason Orendorff that the __enter__ and __exit__ methods could be removed from the context management protocol, and the protocol instead defined directly in terms of the coroutine interface described in PEP 342 (or a cleaner version of that interface with start() and finish() convenience methods) [6]. Guido rejected this idea [7]. The following are some of benefits of keeping the __enter__ and __exit__ methods: - it makes it easy to implement a simple context manager in C without having to rely on a separate coroutine builder - it makes it easy to provide a low-overhead implementation for context managers which don't need to maintain any special state between the __enter__ and __exit__ methods (having to use a coroutine for these would impose unnecessary overhead without any compensating benefit) - it makes it possible to understand how the with statement works without having to first understand the concept of a coroutine 6. Add new references: [4] http://mail.python.org/pipermail/python-dev/2005-July/054658.html [5] http://mail.python.org/pipermail/python-dev/2005-October/056947.html [6] http://mail.python.org/pipermail/python-dev/2005-October/056969.html [7] http://mail.python.org/pipermail/python-dev/2005-October/057018.html 7. Update Example 4 to include a __with__ method: def __with__(self): return self 8. Replace Example 9 with the following example: 9. Here's a proposed native context manager for decimal.Context: # This would be a new decimal.Context method @contextmanager def __with__(self): # We set the thread context to a copy of this context # to ensure that changes within the block are kept # local to the block. This also gives us thread safety # and supports nested usage of a given context. newctx = self.copy() oldctx = decimal.getcontext() decimal.setcontext(newctx) try: yield newctx finally: decimal.setcontext(oldctx) Sample usage: def sin(x): with decimal.getcontext() as ctx: ctx.prec += 2 # Rest of sin calculation algorithm # uses a precision 2 greater than normal return +s # Convert result to normal precision def sin(x): with decimal.ExtendedContext: # Rest of sin calculation algorithm # uses the Extended Context from the # General Decimal Arithmetic Specification return +s # Convert result to normal context -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Fredrik Lundh wrote:
That's not what the decorator is for - it's there to turn the generator used to implement the __with__ method into a context manager, rather than saying anything about decimal.Context as a whole. However, requiring a decorator to get a slot to work right looks pretty ugly to me, too. What if we simply special-cased the __with__ slot in type(), such that if it is populated with a generator object, that object is automatically wrapped using the @contextmanager decorator? (Jason actually suggested this idea previously) I initially didn't like the idea because of EIBTI, but I've realised that "def __with__(self):" is pretty darn explicit in its own right. I've also realised that defining __with__ using a generator, but forgetting to add the @contextmanager to the front would be a lovely source of bugs, particularly if generators are given a default __exit__() method that simply invokes self.close(). On the other hand, if __with__ is special-cased, then the slot definition wouldn't look ugly, and we'd still be free to define a generator's normal with statement semantics as: def __exit__(self, *exc): self.close() Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Nick Coghlan <ncoghlan@gmail.com> writes:
<nit> You don't want to check if it's a generator, you want to check if it's a function whose func_code has the relavent bit set. </nit> Seems a bit magical to me, but haven't thought about it hard. Cheers, mwh -- I think my standards have lowered enough that now I think ``good design'' is when the page doesn't irritate the living fuck out of me. -- http://www.jwz.org/gruntle/design.html

Michael Hudson wrote:
Fair point :)
Seems a bit magical to me, but haven't thought about it hard.
Same here - I'm just starting to think that the alternative is worse, because it leaves open the nonsensical possibility of writing a __with__ method as a generator *without* applying the contextmanager decorator, and that would just be bizarre - if you want to get an iterable, why aren't you writing an __iter__ method instead? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Nick Coghlan wrote:
possibly, but using a decorated __with__ method doesn't make much sense if the purpose isn't to turn the class into something that can be used with the "with" statement.
However, requiring a decorator to get a slot to work right looks pretty ugly to me, too.
the whole concept might be perfectly fine on the "this construct corre- sponds to this code" level, but if you immediately end up with things that are not what they seem, and names that don't mean what the say, either the design or the description of it needs work. ("yes, I know you can use this class to manage the context, but it's not really a context manager, because it's that method that's a manager, not the class itself. yes, all the information that belongs to the context are managed by the class, but that doesn't make... oh, shut up and read the PEP") </F>

Fredrik Lundh wrote:
Heh. OK, my current inclinitation is to make the new paragraph at the end of the "Generator Decorator" section read like this: 4. Add a paragraph to the end of the "Generator Decorator" section: If a generator is used to write a context's __with__ method, then Python's type machinery will automatically take care of applying this decorator. This means that it is just as easy to write a generator-based context manager for a context as it is to write a generator-based iterator for an iterable (see the decimal.Context example below). And then update the decimal.Context example to remove the @contextmanager decorator. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Eric Nieuwland wrote:
It would fail with a TypeError because the relevant slot in the type object was NULL - the TypeError checks aren't shown for simplicity's sake. This behaviour isn't really any different from the existing PEP 343 - the only difference is that the statement looks for a __with__ slot on the original EXPR, rather than looking directly for an __enter__ slot. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
participants (5)
-
Eric Nieuwland
-
Fredrik Lundh
-
Michael Hudson
-
Nick Coghlan
-
Nick Coghlan