[Python-ideas] Cofunctions - Back to Basics

[Steven D'Aprano]

Greg Ewing wrote:
> Still feeling in a peppish mood after the last round of PEP 335,
> I decided to revisit my cofunctions proposal. Last time round, I
> allowed myself to be excessively swayed by popular opinion, and
> ended up with something that lacked elegance and failed to address
> the original issues.
> 
> So, I've gone right back to my initial idea. The current version
> of the draft PEP can be found here:
> 
> http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cd_current/cofunction-pep-rev5.txt 

As per your later email, there is a newer revision here:

http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cd_current/cofunction-pep-rev6.txt


> together with updated examples here:
> 
> http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cd_current/Examples/ 


As a PEP goes, this makes way too many assumptions about the reader's 
knowledge of a small corner of Python. Coroutines themselves are 
relatively new, and hardly in wide-spread use; yield from hasn't even 
made it into a production version of Python yet. But adding new syntax 
and built-ins will effect *all* Python programmers, not just the few who 
use coroutines.

The rationale section is utterly unpersuasive to me. You talk about 
cofunctions being a "streamlined way of writing generator-based 
coroutines", but you give no reason other than your say so that it is 
more streamlined. How about comparing the same coroutine written with, 
and without, a cofunction?

You also claim that this will allow "the early detection of certain 
kinds of error that are easily made when writing such code", but again 
no evidence is given. You should demonstrate such an error, and the 
"hard-to-diagnose symptoms" it causes, and some indication whether these 
errors happen in practice or only in theory.

The Cofunction Definitions section does not help me understand the 
concept at all:

(1) You state that it is a "special kind of generator", but don't give 
any clue as to how it is special. Does it do something other generators 
can't do? A cofunction is spelled 'codef' instead of 'def', but what 
difference does that make to the behaviour of the generator you get? How 
does it behave differently to other generators?

(2) Cofunctions, apparently, "may" contain yield or yield from. 
Presumably that means that yield is optional, otherwise it would be 
"must" rather than "may". So what sort of generator do you get without a 
yield? The PEP doesn't give me any clue.

If I look at the external examples (the PEP doesn't link to them), I see 
plenty of examples of syntax, but nothing about semantics. E.g.:

     codef parse_elem(opening_tag):
         name = opening_tag[1:-1]
         closing_tag = "</%s>" % name
         items = parse_items(closing_tag)
         return (name, items)

What does this actually do?


(3) Something utterly perplexing to me:

"From the outside, the distinguishing feature of a cofunction is that it
cannot be called directly from within the body of an ordinary function. 
An exception is raised if such a call to a cofunction is attempted."

Many things can't be called as functions: ints, strings, lists, etc. It 
simply isn't clear to me how cofunctions are different from any other 
non-callable object. The PEP should show some examples of what does and 
doesn't work. In particular, the above as stated implies the following:

def spam():
     x = cofunction()  # fails, since directly inside a function

x = cofunction()  # succeeds, since not inside a function

Surely that isn't what you mean to imply, is it?


Is there any prior art in other languages? I have googled on 
"cofunction", and I get many, many hits to the concept from mathematics 
(e.g. sine and cosine) but virtually nothing in programming circles. 
Apart from this PEP, I don't see any sign of prior art, or that 
cofunction is anything but your own term. If there is prior art, you 
should describe it, and if not, you should say so.

Based on the PEP, it looks to me that the whole premise behind 
"cofunction" is that it allows the user to write a generator without 
using the word yield. That *literally* the only thing it does is replace 
a generator like this:

def coroutine():
     yield from f()

with

codef coroutine():
     f()

with some syntactic sugar to allow Python to automagically tell the 
difference between ordinary function calls and cofunctions inside a 
cofunction: "Do What I Mean" functionality.

If cofunctions actually are more than merely a way to avoid writing 
yield, you should explain how and why they are more. You should also 
explain why the form

     yield from f()  # function call

is so special that it deserves a new keyword and a new built-in, while 
this otherwise semantically identical call:

     x = f()
     yield from x  # here's one we prepared earlier

does not.


In the Motivation and Rationale section, you state:

     If one forgets to use ``yield from`` when it should have
     been used, or uses it when it shouldn't have, the symptoms
     that result can be extremely obscure and confusing.

I'm not sympathetic to the concept that "remembering which syntax to use 
is hard, so let's invent even more syntax with non-obvious semantics".

I don't believe that remembering to write ``codef`` instead of ``def`` 
is any easier than remembering to write ``yield from`` instead of 
``yield`` or ``return``.



-- 
Steven