Callbacks to generators

Dominic Fox at
Tue Jun 8 15:49:06 CEST 2004

It would be nice if something like this *could* work - you could instantly
and effortlessly transform a SAX parser into a .Net-style XmlReader. Alas,
it cannot...

"yield" in a function causes the function to return a generator object to
the caller - in this case, the caller of on_token, not the caller of
process_iter. The inner function "on_token" in "process_iter" returns a
generator when it is called by "process", and "process" promptly throws this
generator away. The value returned by "process_iter" is just the return
value from "process".

It might be possible to do something with call/cc, but of course Python
doesn't have that.


-----Original Message-----
From: Humpty Dumpty [mailto:oliver.schoenborn at] 
Sent: 08 June 2004 13:07
To: python-list at
Subject: Re: Callbacks to generators

> def process(text, on_token):
>     ...
> For each token that "process" finds in "text", it creates a token object,
> "token", and calls "on_token(token)".
> Now, suppose I wanted to create a generator based on this function. I
> to do the following:
> def process_iter(text):
>     def on_token(token):
>         yield token
>     process(text, on_token)
> However, rather than create a generator as I had hoped, this function
> doesn't return anything. I guess it creates a bunch of singleton
> one per token, and throws them away. In any case, is there a way to do
> I am trying to do here without resorting to threads?

Something weird here. I hven't used generators much, but seems to me that:

1) you maybe don't need them:

 def process_iter(text):
     def on_token(token):
         return token
     process(text, on_token)

2) you need some sort of while loop so the fnction doesn't return after the
first yield:

 def process_iter(text):
     def on_token(token):
         while 1:
            yield token
     process(text, on_token)

I'm curious to understand if neither of those are true, why...


More information about the Python-list mailing list