pickling generators

[Gerrit Holl]

Paul Boehm schreef op zaterdag 22 maart om 14:58:49 +0000:
> our problem, however, is that we can't properly do savegames with this design,
> as generator instances are not pickle- and unpickleable.

True.

> we're now looking for options, and advice, on the feasability
> of getting support for this into future python versions, either by learning
> how to do it, and implementing it ourselves, or by finding someone willing
> to help us out.

I have written some code to do this. It assumes:

    * When the same arguments are passed to the generators function, it
      yields the same values,
    * No value is yielded twice, or it is yielded forever,
    * Requires Python 2.3.

My __getstate__ contains:

            if isinstance(d[attr], types.GeneratorType):
                g = d[attr]
                try:
                    # dropwhile includes, so this is correct
                    latest = g.next()
                except StopIteration: # no need for pickling
                    pass
                else:
                    fm = g.gi_frame
                    # XXX: is this safe?
                    func = fm.f_globals[inspect.getframeinfo(fm)[2]]
                    argnames, _, _, argvalues = inspect.getargvalues(fm)
                    args = tuple([argvalues[a] for a in argnames])
                    d[attr] = (func, args, latest)

My __setstate-_ contains:

           if isinstance(val, tuple) and len(val) == 3 and callable(val[0]):
                func, args, latest = val
                def f(x): return x != latest
                g = func(*args)
                newg = itertools.dropwhile(f, g)
                setattr(self, key, newg)

It may me possible to do this better, however; I'm eager to hear about that!

You can also use a class iterator with a __getstate__ and __setstate__ to
solve this problem.

yours,
Gerrit.

-- 
230. If it kill the son of the owner the son of that builder shall be
put to death. 
        -- Hammurabi, Code of Law
--
Asperger Syndroom - een persoonlijke benadering:
	http://people.nl.linux.org/~gerrit/
Het zijn tijden om je zelf met politiek te bemoeien:
	http://www.sp.nl/