Destruction of generator objects

[Kay Schluehr]

On Aug 11, 2:50 pm, Stefan Bellon <sbel... at sbellon.de> wrote:

> So why is the destructor not called when the generator is even
> explicitly 'del'ed? Does somebody else still hold a reference on it?

You ( we ) have produced a reference cycle. In that case __del__
doesn't work properly ( according to the docs ). The cycle is caused
by the following assignment in your code:

   self.gen = self._value()

So we have to refactor the solution to eliminate the cycle. I spent
some time to create a generic decorator solution and a protocol for
handling / releasing the resources.

1) Create a FinalizerGenerator class that is cycle free ( of course
you can find a trick to shoot yourself in your foot ). Pass a
generator function together with its arguments into the constructor.
An additional callable attribute close is used together with __del__.

class FinalizerGenerator(object):
    def __init__(self, gen, *args, **kwd):
        print "gen init"
        self._gen = gen    # no cycle when passing the _value function
        self.close = lambda: sys.stdout.write("gen del")  # assign
cleanup function

    def __del__(self):
        self.close()

    def __iter__(self):
        print "gen iter"
        return self

    def next(self):
        print "gen next"
        return self.gen.next()

2) Define generators s.t. they yield the resource to be destroyed as
their first value.

def producer(resource):
    print "gen value"
    yield resource       # yield resource before start iteration
    for item in resource:
        yield item

3) Define the destructor for the resource. The resource must be passed
as a first argument. The return value is a callable without arguments
that serves as a close() function within FinalizerGenerator.__del__
method.

def close_resource(resource):
    return lambda: sys.stdout.write("close resource: %s"%resource)

4) The finalize_with decorator

def finalize_with(close_resource):
    def closing(func_gen):
        def fn(*args, **kwd):
            # fg serves as a replacement for the original generator
func_def
            fg = FinalizerGenerator(func_gen)(*args, **kwd)
            # apply the closing protocol
            resource = fg.next()
            fg.close = close_resource(resource)
            return fg
        fn.__name__ = func_gen.__name__
        return fn
    return closing


5) decorate the generator and check it out

@finalize_with(close_resource)
def producer(resource):
    print "gen value"
    yield resource
    for item in resource:
        yield item

def test():
    pg = producer([1,2,3])
    pg.next()

>>> test()
gen init
gen next        # request for resource in finalize_with
gen value
gen next
close resource: [1, 2, 3]  # yep