[Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)

[Nick Coghlan]

Phillip J. Eby wrote:
> At 10:51 AM 4/21/2006 -0400, A.M. Kuchling wrote:
>> On Fri, Apr 21, 2006 at 07:31:35PM +1000, Nick Coghlan wrote:
>>> fit the new definition. So we settled on calling them "context managers"
>>> instead.
>>  ...
>>> method. Instead, the new term "manageable context" (or simply "context")
>>> was introduced to mean "anything with a __context__ method". This was OK,
>> Meaning that 'manageable context' objects create and destroy 'context
>> managers'...  My view is still that 'context manager' is a terrible
>> name when used alongside objects called 'contexts': the object doesn't
>> manage anything, and it certainly doesn't manage contexts -- in fact
>> it's created by 'context' objects.
> 
> And that's more or less why I wrote the documentation the way I did.
> 
> Nick, as I understand your argument, it's that we were previously using the 
> term "context manager" to mean "thing with __enter__ and __exit__".  But 
> that was *never* my interpretation.
> 
> My understanding of "context manager" was always, "thing that you give to a 
> with statement".

Then why didn't you speak up when the discussion was summarised in PEP 343 for 
Guido's approval? I said it explicitly:

      This PEP proposes that the protocol used by the with statement be
      known as the "context management protocol", and that objects that
      implement that protocol be known as "context managers". The term
      "context" then encompasses all objects with a __context__() method
      that returns a context manager object. (This means that all context
      managers are contexts, but not all contexts are context managers)

I guess a slight ambiguity came in from the fact I didn't spell out that the 
protocol I was referring to was all three methods with __context__ returning 
self (i.e. the moral equivalent of the 'iterator protocol'). But the rest of 
the paragraph doesn't make any sense otherwise.

Under Resolved Issues, before the recent changes, it said this:

  3. After this PEP was originally approved, a subsequent discussion
        on python-dev [4] settled on the term "context manager" for
        objects which provide __enter__ and __exit__ methods, and
        "context management protocol" for the protocol itself. With the
        addition of the __context__ method to the protocol, the natural
        adjustment is to call all objects which provide a __context__
        method "contexts" (or "manageable contexts" in situations where the
        general term "context" would be ambiguous).
        This is now documented in the "Standard Terminology" section.

*This* is what Guido approved, not what is currently written up in the PEP on 
python.org.

> So to me, when we added a __context__ method, we were creating a *new 
> object* that hadn't existed before, and we moved some methods on to 
> it.  Thus, "context manager" still meant "thing you give to the with 
> statement" -- and that never changed, from my POV.

That may have been what you personally thought, but it's not what the PEP 
said. If you disagreed with the summarisation in the PEP, you should have said 
so before Guido approved it, or brought it back to python-dev as a discussion 
about changing the standard terminology rather than just "the PEP's confusing, 
I want to clear it up" (and completely changing the meaning in the process).

> And that's why I see the argument that we've "reversed" the terminology as 
> bogus: to me it's been consistent all along.  We just added another object 
> *besides* the context manager.

I agree with the second part, but not the first part. Originally we only had 
context managers (objects that managed their own state in their 
__enter__/__exit__ methods). Jason brought up the point that this excluded 
decimal.Context objects because it was extremely difficult to produce a 
thread-safe and nesting-safe __enter__/__exit__ pair to manipulate the decimal 
context. Without a __context__ method, the decimal module would have had to 
provide a separate object for use as a context manager.

So we added contexts to the PEP - objects that could produce a context manager 
to work in conjunction with the with statement to manage the state of the 
context object. Context managers created in this fashion, instead of operating 
on their own state, actually operate on the state of the context that created 
them.

This is what got reversed in the contextlib docs - the PEP said that context 
managers work on contexts the same way that iterators work on iterables.The 
contextlib docs (and the latest version of the PEP) say that contexts 
manipulate context managers.

This is just plain bad English. "context" is a passive noun like "iterable" - 
it doesn't imply any sort of active operation. "context manager" on the other 
hand describes an actor doing something, just like "iterator" does.

> 
> Note too that the user of the "with" statement doesn't know that this other 
> object exists, and in fact sometimes it doesn't actually exist, it's the 
> same object.  None of this is relevant for the with-statement user, only 
> the context manager.  So there's no reason (IMO) to monkey with the 
> definition of "context manager" as "thing you use in a with statement".

Paraphrasing:

  Note too that the user of the "for" statement doesn't know that this other
  object exists, and in fact sometimes it doesn't actually exist, it's the
  same object.  None of this is relevant for the for-statement user, only
  the iterator writer.  So there's no reason (IMO) to monkey with the
  definition of "iterator" as "thing you use in a for statement".

"context" is to "context manager" as "iterable" is to "iterator". Why is this 
a difficult concept? The only difference is that we have a builtin to do the 
obj.__iter__() call, but have to do obj.__context__() explicitly.

And while "thing you use in a with statement" may have been the definition of 
context manager in your mind, it was never the definition in the PEP.

> Now, I get your point about @contextmanager on a __context__ method, and I 
> agree that that seems backwards at first.  What I don't see is how to 
> change the terminology to handle that subtlety in a way that doesn't muck 
> up the basically simple definitions that are in place now.

Easy: we go back to the definitions we used in the originally approved PEP, 
where "context" precisely paralleled "iterable" and "context manager" 
precisely paralleled "iterator".

Leverage off people's long experience with iterators and iterables instead of 
doing something deliberately different.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org