On Mon, 18 Nov 2019 at 18:34, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
On Mon, 18 Nov 2019 at 17:46, Paul Moore <p.f.moore@gmail.com> wrote:
What about
f = open(filename) header = f.readline() with f: # use f
I would naturally rewrite that as
with open(filename) as f: header = f.readline() # use f
which would work just as well with opened instead of open. The opened function returns a context manager whose __enter__ method returns the file object which then has the corresponding file object methods.
My apologies. I'm giving examples that I intend to suggest the issue I'm concerned about, but don't actually demonstrate them very well (because they are "toy" examples, and I'm rushing to put something together and not taking the time to give a better crafted example). By doing so I'm wasting everyone's time, as they show corrections that work with my toy cases, but leave me feeling "but that wasn't the point" (when I clearly hadn't explained the point well enough). I should know better than to do this - sorry.
I think that nested was fine but in combination with open it was prone to misuse. By the time with/contextlib etc had shipped in 2.5 it was easier to blame nested (which also had other flaws) so it took the fall for open.
I think that nested made assumptions about how context managers worked that weren't always true in practice. That's acceptable (if a little risky) as long as it's easy to know its limitations and not use it with inappropriate CMs. Unfortunately, it was extremely tempting to use it with open() which didn't satisfy the additional requirements that nested() imposed, and that made it an attractive nuisance. Treating the problem as being caused by nested() not working properly with *all* context managers was relatively easy, as nested was new. Trying to rework open to fit the expectations of nested would have been far harder (because open has been round so long that backward compatibility is a major issue), as well as not helping in the case of any *other* CMs that didn't work like nested expected them to.
Anyway, I already said that where you choose to draw the line over what a context manager is (assuming you feel that the current definition is wrong), depends on your perspective.
It's not so much that the definition is wrong. It just isn't really defined and that makes it difficult to do anything fancy with multiple context managers. You need protocol contraints on both sides to be able to build useful utilities/patterns. The bar set for nested was that it should be able to recover from errors before it even gets called!
The definition is clearly and precisely implied by the PEP and the semantics of the with statement. A CM is anything that has __enter__ and __exit__. See https://www.python.org/dev/peps/pep-0343/#standard-terminology. Maybe it "makes it difficult to do anything fancy with multiple context managers" - I don't know about that. It feels like a fairly large generalisation to get from "nested doesn't work the way we'd like it to" to that statement. But I haven't really thought about the problem much. In my experience, I don't find much need for fancy context manager combinations. I mostly just do "with <something>: <do some work>". I might refactor a complex set of with statements, but I'd do that in an application specific function, not by trying to design a generalised CM combinator. I sympathise with the interest in writing clever combinators. I just don't know that people are hurting from the lack of them. It's probably worth reiterating that this whole sub-thread is for me nothing more than interesting but entirely theoretical speculation - I don't see a need for *any* solution to the original problem of writing multiple CMs in a single with statement "more readably" than the existing approach of using backslashes, so there's no actual problem being solved by the proposals here (unless other use cases come up in the discussion - and I don't miss nested(), so rehabilitating that isn't compelling for me either).
A PEP seems premature as I'm not sure I have any clear solution...
It's certainly premature to write one at this point. But all I was saying is that if you intend to redefine a standard term like "context manager" that was itself defined in a PEP, you'll need a new PEP to do so at some point. Paul