On Mon, 18 Nov 2019 at 15:54, Paul Moore <p.f.moore@gmail.com> wrote:
On Mon, 18 Nov 2019 at 11:12, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
I am proposing the root of the problem here is the fact that open acquires its resource (the opened file descriptor) before __enter__ is called. This is what I mean by a context manager that "misbehaves". If there was a requirement on context managers that __exit__ cleans up after __enter__ and any resource that needs cleaning up should only be acquired in __enter__ then there would never have been a problem with nested. [...] What I am saying is that conceived as a context manager the object returned by open misbehaves. I think that not just nested but a number of other convenient utilities and patterns could have been possible if opened has been used instead of open and if context managers were expected to meet the constraint: """ There should be no need to call __exit__ if __enter__ has not been called. """ Of course a lot of time has passed since then and now there are probably many other misbehaving context managers so it might be too late to do anything about that.
Hi Oscar, Thanks for the explanation. I see what you mean now, and that *was* something I got from the previous discussion, it's just that I guess I'm so used to the current behaviour that I never really thought of it as "misbehaviour". I'm not 100% convinced that there aren't edge cases where even your strengthened requirements on a context manager might not be enough. For example, if __enter__ is called, but raises an exception, is calling __exit__ required then?
It has never been the case that __exit__ would be called if __enter__ does not exit successfully even for the basic form of the with statement e.g.: class ContextMgr: def __enter__(self): print('Entering...') raise ValueError('Bad stuff') def __exit__(self, *args): print('Exiting') with ContextMgr(): pass Gives $ python f.py Entering... Traceback (most recent call last): File "f.py", line 8, in <module> with ContextMgr(): File "f.py", line 4, in __enter__ raise ValueError('Bad stuff') ValueError: Bad stuff You can also see this in the original specification of the with statement since __enter__ is called outside the try suite: https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement
Consider
@contextmanager def open_2_files(): f = open("file1") g = open("file2") try: yield (f,g) finally: g.close() f.close()
That meets your criterion, but if open("file2") fails, you're still in a mess. Of course, that's a toy example, and could be written to fix that,
That example is a poor context manager by anyone's definition and can easily be fixed: @contextmanager def open_2_files(): with open('file1') as f: with open('file2') as g: yield (f, g)
and we could even close that loophole by saying "a context manager should only manage one resource", but we can probably carry on down that route for quite a while (and "should only manage one resource" is not actually correct - the whole *point* of something like nested() would be to manage multiple resources).
I don't see why you would say that managing multiple resources is a problem here. It's a question of who is responsible for what. The context manager itself is responsible for cleaning up anything if an exception is raised *inside* it's __enter__ and __exit__ methods. Once the manager returns from __enter__ though it hands over control. Then the with statement and other supporting utilities are responsible for ensuring that __exit__ is called at the appropriate later time. The problem with a misbehaving context manager is that it creates a future need to call __exit__ before it has been passed to a with statement or any other construct that can guarantee to do that. -- Oscar