Chris Barker wrote:
Nick Coghlan wrote:
Chris, would you be open to trying a thought experiment with some of your students looking at ways to introduce function-scoped deterministic resource management *before* introducing with statements?
I'm with Chris, I think: this seems inappropriate to me. A student has to be rather sophisticated to understand resource management at all in Python. Eg, generators and closures can hang on to resources between calls, yet there's no syntactic marker at the call site.
The idea here is to change the requirement for new developers from "telling the interpreter what to *do*" (which is the situation we have for context managers) to "telling the interpreter what we *want*" (which is for it to link a managed resource with the lifecycle of the currently running function call, regardless of interpreter implementation details)
I think this attempt at a distinction is spurious. On the syntactic side, with open("file") as f: results = read_and_process_lines(f) the with statement effectively links management of the file resource to the lifecycle of read_and_process_lines. (Yes, I know what you mean by "link" -- will "new developers"?) On the semantic side, constructs like closures and generators (which they may be cargo- culting!) mean that it's harder to link resource management to (syntactic) function calls than a new developer might think. (Isn't that Nathaniel's motivation for the OP?) And then there's the loop that may not fully consume an iterator problem: that must be explicitly decided -- the question for language designers is which of "close generators on loop exit" or "leave generators open on loop exit" should be marked with explicit syntax -- and what if you've got two generators involved, and want different decisions for both? Chris:
I can see that, but I'm not sure newbies will -- it either case, you have to think about what you want -- which is the complexity I'm trying to avoid at this stage.
Indeed.
Until much later, when I get into weak references, I can pretty much tell people that python will take care of itself with regards to resource management.
I hope you phrase that very carefully. Python takes care of itself, but does not take care of the use case. That's the programmer's responsibility. In a very large number of use cases, including the novice developer's role in a large project, that is a distinction that makes no difference. But the "close generators on loop exit" (or maybe not!) use case makes it clear that in general the developer must explicitly manage resources.
That's what context mangers are for, in fact. YOU can use:
with open(...) as infile: .....
Without needing to know what actually has to be "cleaned up" about a file. In the case of files, it's a close() call, simple enough (in the absence of Exceptions...), but with a database connection or something, it could be a lot more complex, and it's nice to know that it will simply be taken care of for you by the context manager.
But somebody has to write that context manager. I suppose in the organizational context imagined here, it was written for the project by the resource management wonk in the group, and the new developer just cargo-cults it at first.
The big refactoring benefit that this feature would offer over with statements is that it doesn't require a structural change to the code - it's just wrapping an existing expression in a new function call that says "clean this up promptly when the function terminates, even if it's still part of a reference cycle, or we're not using a reference counting GC".
hmm -- that would be simpler in one sense, but wouldn't it require a new function to be defined for everything you might want to do this with? rather than the same "with" syntax for everything?
Even if it can be done with a single "ensure_cleanup" function, Python isn't Haskell. I think context management deserves syntax to mark it. After all, from the "open and read one file" scripting standpoint, there's really not a difference between f = open("file") process(f) and with open("file") as f: process(f) (see "taking care of Python ~= taking care of use case" above). But the with statement and indentation clearly mark the call to process as receiving special treatment. As Chris says, the developer doesn't need to know anything but that the object returned by the with expression participates "appropriately" in the context manager protocol (which she may think of as the "with protocol"!, ie, *magic*) and gets the "special treatment" it needs. So (for me) this is full circle: "with" context management is what we need, but it interacts poorly with stateful "function" calls -- and that's what Nathaniel proposes to deal with.