[Python-ideas] Cofunctions - an enhancement to yield-from

Nick Coghlan ncoghlan at gmail.com
Mon Aug 2 14:03:39 CEST 2010


On Mon, Aug 2, 2010 at 10:16 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 02 Aug 2010 12:07:06 +1200
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> There are still use cases for
>> 'yield from', when you're dealing with generators that are
>> designed to produce values.
>
> It was the tree-walking example, right? It really looked like it solved
> a non-existing problem.

For generators, the important benefit of PEP 380 is that it makes sure
that nested cleanup happens in the right order.

Consider the following currently valid (but subtly broken) generator pair:

  def receive_message_components(channel):
      with channel.open() as session:
          while 1:
              data = session.wait_for_next()
              yield data
              if data == channel.EOM:
                  break

  def receive_multiple_message_components(server_details,
channel_details, limit=None):
      with Server(server_details) as server:
          channel = server.channel(channel_details)
          n = 0
          while 1:
              for component in receive_message_components(channel):
                  yield component
              if limit is not None:
                  n += 1
                  if n >= limit:
                      break

That code is most likely broken: if an exception (e.g. GeneratorExit)
gets thrown into the outer generator, the server connection will be
closed while an open session is still using that connection (since the
inner generator doesn't get closed until the outer generator's
reference to it gets released, by which time the with statement will
have killed the server connection). However, if the body of the inner
generator were written inline in the outer generator instead then
everything would be fine - the session would be closed before the
server connection because the exception handling would correctly
propagate out from the innermost yield. PEP 380 makes it easy to
factor out subgenerators without needing to worry about subtle
misbehaviours of exception handling due to the delayed closure of the
subgenerators:

  def receive_multiple_message_components(server_details,
channel_details, limit=None):
      with Server(server_details) as server:
          channel = server.channel(channel_details)
          n = 0
          while 1:
              yield from receive_message_components(channel)
              if limit is not None:
                  n += 1
                  if n >= limit:
                      break

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia



More information about the Python-ideas mailing list