[Python-Dev] Context manager lifetime rules Was: Re: Adding bytes.frombuffer() constructor to PEP 467

[Hans-Peter Jansen]

Hi Yury,

adjusted subject, since I'm dragging the discussion away from it.

On Donnerstag, 5. Januar 2017 20:28:26 Yury Selivanov wrote:
> On 2017-01-05 7:11 PM, INADA Naoki wrote:
> >> bytes.frombuffer(x) is bytes(memoryview(x)) or memoryview(x).tobytes().
> > 
> > There is pitfall: memoryview should be closed.
> > So b = bytes.frombuffer(x) is:
> > 
> > with memoryview(x) as m:
> >      b = bytes(m)
> >      # or b = m.tobytes()
> 
> Thinking more about this, and after looking at my own code in asyncpg
> and uvloop, I'm now in favor of adding bytes.frombuffer() with the
> proposed signature: ``bytes.frombuffer(byteslike, length=-1, offset=0)``
> 
> Inada-san is right, the memoryview should be explicitly released, but
> few do that. Instead, a lot of people simply rely on CPython refcounting
> semantics, which will cause the temporary memoryview be GCed asap.  That
> won't work so flawlessly in PyPy and will cause hard to understand bugs.

Did you noticed, that we're stirring in the same kind of soup lately?
(apart from my person, who's not that deep in the details)

Given above code snippet, my issue is caused from "m" surviving *too* long 
after the context manager block ended, resulting in a delayed release of the 
memoryview, which in turn interferes with subsequent code.

Simple minded, as I am, I would expect, that "m" is actively removed from 
local context as soon as the context manager block ends. I would even argue, 
that this is part of the contract, that the context manager offers here.

AFAICS, "m" is handed over the the GC, but:
	* this is somewhat surprising for the (probably simple minded) programmer
   * leads to problems *already*

A workaround is deleting "m" actively:

    with memoryview(x) as m:
        b = bytes(m)
    del m

I would like to discuss the rationals of this construct from a language design 
POV, not language implementation POV.

usually-hacking-del-free-code-ly y'rs,
Pete