[Python-ideas] Deterministic iterator cleanup

[Chris Barker]

On Sat, Oct 22, 2016 at 8:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:


> Pondering this overnight, I realised there's a case where folks using
> Python primarily as a scripting language can still run into many of
> the resource management problems that arise in larger applications:
> IPython notebooks



> This is likely mitigated in practice *today* by IPython users mostly
> being on CPython for access to the Scientific Python stack,


sure -- though there is no reason that Jupyter notebooks aren't really
useful to all sort of non-data-crunching tasks. It's just that that's the
community it was born in.

I can imagine they would be great for database exploration/management, for
instance.

Chris, would you be open to trying a thought experiment with some of

your students looking at ways to introduce function-scoped
> deterministic resource management *before* introducing with
> statements?


At first thought, talking about this seems like it would just confuse
newbies even MORE. Most of my students really want simple examples they can
copy and then change for their specific use case.

But I do have some pretty experienced developers (new to Python, but not
programming) in my classes, too, that I might be able to bring this up with.

     # Cleaned up whenever the interpreter gets around to cleaning up

> the function locals
>     def readlines_with_default_resource_management(fname):
>         return open(fname).readlines()
>
>     # Cleaned up on function exit, even if the locals are still
> referenced from an exception traceback
>     # or the interpreter implementation doesn't use a reference counting GC
>     from local_resources import function_resource
>
>     def readlines_with_declarative_cleanup(fname):
>        return function_resource(open(fname)).readlines()
>
>     # Cleaned up at the end of the with statement
>     def readlines_with_imperative_cleanup(fname):
>         with open(fname) as f:
>             return f.readlines()
>
> The idea here is to change the requirement for new developers from
> "telling the interpreter what to *do*" (which is the situation we have
> for context managers) to "telling the interpreter what we *want*"
> (which is for it to link a managed resource with the lifecycle of the
> currently running function call, regardless of interpreter
> implementation details)
>

I can see that, but I'm not sure newbies will -- it either case, you have
to think about what you want -- which is the complexity I'm trying to avoid
at this stage. Until much later, when I get into weak references, I can
pretty much tell people that python will take care of itself with regards
to resource management.

That's what context mangers are for, in fact. YOU can use:

with open(...) as infile:
    .....

Without needing to know what actually has to be "cleaned up" about a file.
In the case of files, it's a close() call, simple enough (in the absence of
Exceptions...), but with a database connection or something, it could be a
lot more complex, and it's nice to know that it will simply be taken care
of for you by the context manager.

The big refactoring benefit that this feature would offer over with
> statements is that it doesn't require a structural change to the code
> - it's just wrapping an existing expression in a new function call
> that says "clean this up promptly when the function terminates, even
> if it's still part of a reference cycle, or we're not using a
> reference counting GC".


hmm -- that would be simpler in one sense, but wouldn't it require a new
function to be defined for everything you might want to do this with?
rather than the same "with" syntax for everything?

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20161024/c27aad79/attachment.html>