Whoops I sent too soon. I'll try again...
On Tue, 19 Nov 2019 at 11:01, Oscar Benjamin
On Tue, 19 Nov 2019 at 08:21, Paul Moore
wrote: On Mon, 18 Nov 2019 at 23:42, Oscar Benjamin
wrote: Context managers and the with statement are all about assigning responsibility so there needs to be an understanding of what has responsibility for what at any given time.
Somewhere (I can't recall where) there's an example of context managers that allow you to do something like
with html(): with body(): with div(class="main-panel"): etc...
Whether that counts as a clever usage, or an abuse, is somewhat a matter of opinion.
I think that's fine as a use of context managers. I assume that the intention is that the CM returned by html() is responsible for adding both the opening and closing html tags. It isn't clear to me from the example how errors are expected to be handled though - would you still want to add the closing tag or would you just catch the error higher up and return a completely different error document to the user.
I'm also not clear whether the rules you are proposing would allow or prohibit such a use.
No one is proposing to prohibit such a use! I guess it's unclear what is being proposed because nothing specific has been proposed...
I'm also not clear what nested(html(), body()) might mean.
with nested(html(), body()): ... is intended to be equivalent to with html(): with body(): ... The difference between them is what happens if body() raises before returning a CM. What I am saying is that it shouldn't be nested's responsibility to handle errors at the stage where the context managers are being created (before nested is called). The idea that nested should be able to handle that or is otherwise flawed in design comes from the fact that there are context managers like open() that are expected to fail before __enter__ is called. If you (the author of the above) care about that then you should write your context managers so that the opening and closing tags are emitted in __enter__ and __exit__ rather than the initial call to html(). Note that this happens automatically for you if you use the contextmanager decorator which is the obvious way to implement the above: @contextmanager def html(): print('<html>') yield print('</html>')
Conceded, that's a general statement not a specific rule. But the rule you gave earlier:
If there was a requirement on context managers that __exit__ cleans up after __enter__ and any resource that needs cleaning up should only be acquired in __enter__ then there would never have been a problem with nested.
doesn't really apply here, as I don't think of the HTML example in terms of "resources".
Perhaps resources is not a generally applicable term here.
To look at this another way, using the example of open again, what open() returns isn't an object that manages a resource - it's the resource *itself* (the open file). The fact that the resource can be self-managing at all is a result of the fact that the CM protocol and the with statement are defined purely in terms of a mechanism. Open file objects aren't closed using __exit__(), they are closed using close(). But by making __enter__() do nothing and __exit__() call close, you can make file objects work with the with statement. But you can't make a file object's __enter__ method "acquire any resource that needs cleaning up", because the __enter__ is a method on that resource in the first place.
So your proposal is really saying that self-managing resources are disallowed, and all resources need a *pair* of classes - one to represent the resource, and one to represent a "manager" for that resource. That's possible (that's what the opened() example in the with statement PEP did) but it's more restrictive than the current context manager protocol. The restrictions allow you do make more assumptions, and hence write certain functions that you otherwise couldn't, but do the benefits justify the costs?
I think that at this stage it is not clear if the benefits justify the costs of any change but at the beginning it would have been better to define expectations more clearly. If I was to propose anything here it would not be to disallow anything that you can currently do with context managers. Rather the suggestion would be to: 1. Clearly define what a well-behaved context manager is. 2. Add convenient utilities for working with well behaved context managers. 3. Add well-behaved alternatives for open and maybe others. 4. Add Random832's utility for adapting misbehaving context managers. The point is not that anything should be disallowed but that we can have useful context manager utilities if we have a clear understanding of how to use them and easy ways to use them correctly. Then if someone hits up against a bug from using a misbehaving context manager where they shouldn't the response can be "change the context manager or don't use it in that situation" rather than "that's a bug in the useful utilities so let's remove those". -- Oscar