[Python-ideas] Proposal for function expressions

[Chris Perkins]

I have a proposal for a language feature - a limited form of function-
definition expression, similar (superficially) to Ruby's blocks.

The big problem with making "def" an expression is the indentation of
the function body. You can't just embed an indented block of code into
the middle of an expression. The solution I propose (inspired by Ruby)
is to allow a block of code to be appended to the end of an expression
in certain, limited circumstances. The block is simply syntactic sugar
for a local anonymous function definition which is passed as an
argument to the function call that it follows.

First, a simple example to give the broad strokes:
foo() do:
    BODY

is equivalent to:

def ANON():
    BODY
foo(ANON)

where the name ANON is a placeholder for illustration - it is never
actually bound in the local namespace.

Specifically, here is what I propose:

* A new syntactic construct, the "block", which consists of:
  * the keyword "do" (alternative: "def", repurposed)
  * an optional argument list
  * a colon
  * an indented block
* A block is only allowed immediately following a call (or indexing
expression? see below), on the same line as the closing right brace
* The block defines an anonymous local function which is passed as an
argument to the call that it follows.
* The block is passed as the last argument to the function by default,
but this can be changed by putting a placeholder, the character "&",
into the parameter list of function. This is best illustrated by
example:

# 1) In the following, foo recieves TWO arguments: 23 and a callable,
in that order.
foo(23) do:
    pass

# 2) Here, foo also recieves TWO arguments: a callable and 23, in that
order.
foo(&, 23) do:
    pass

Why "&"? No particular reason - just because Ruby uses "&" for
something similar (but not the same).
Why do we need the "&" feature?  Consider this example:

map(&, my_list) do(item):
    return do_something_to(item)

This also works for keyword arguments:

foo(a=3, b=&, c=7) do():
    whatever()


To make this syntax work, we need several restrictions on where a
block is allowed. Intuitively, the rules are:
* If the line following "do" would normally be indented, then a block
is not allowed.
* If the line following the "do" would be one on which leading
indentation is insignificant, a block is not allowed.

To clarify, the first rule means that this is not allowed:
if foo() do:
    # are we in the body of the block, or of the if?

The second rule means that this is not allowed:
bar(23, foo() do:
    body_of_block()
    ) # closing brace of call to bar


Here are some properties of blocks that may not be immediately
obvious:
* Blocks are a feature of the call site, and do not affect function
definitions. In other words, there is no such thing as a "function
expecting a block", as there is in Ruby. From the callee's point of
view, he has simply been passed a callable as an argument.
* Blocks can be used with any existing callable that expects to be
passed a function as one of its arguments.


OK, let's move on to the fun part: Motivating Examples.

###################
# Networking
dfr = twisted.whatever(...)
dfr.addCallback() do(result):
    handle(result)
dfr.addErrback() do(err):
    handle_err(err)


###################
# GUI (for some hypothetical library)
b = my_widget.add_button('Go')
b.on('click') do(evt):
    if evt.ctrl_key:
        do_something()
    else:
        do_other_stuff()


###################
# Routing HTTP requests
map.match('/') do(req):
    return render_home(req)
map.match('/about') do(req):
    return render_about(req)
map.match(/(?P<controller>\w+)\/$/) do(req, **kw):
    return handlers[kw['controller']](req)


###################
# Threads
thread.start_new() do:
    do_some_work()


###################
# Reduce the temptation to cram too much into a lambda.

# Sort a list of filenames that look like this:
#   Xyz-1.1.3.tgz, acbd-4.7.tgz, Xyz-1.2.5.tgz, ...
my_list.sort(key=&, reverse=True) do(item):
    name, _ = os.path.splitext(item)
    root, _, version = name.partition('-')
    parts = version.split('.')
    return (root.lower(),) + tuple(map(int, parts))

stuff = filter(&, my_list) do(item):
    # some code here...
    return pred(item)

stuff = re.sub(rx, &, s) do(m):
	t = m.group(0)
    if cond(t): return t.upper()
    else if othercond(t): return t.lower()
    else: return ''


###################
# Faking switch
switch(expr) do(case):
    case(int) do:
        stuff()
    case(str, bytes) do:
        other_stuff()

switch(value) do(case):
    case[0:10] do(val):
        handle_small(val)
    case[10:100] do(val):
        handle_medium(val)
    case.default() do(val):
        handle_big(val)


###################
# Overloaded functions (adapted from PEP 3124)
@overload
def flatten(when):
    when(basestring) do(ob):
        yield ob
    when(Iterable) do(ob):
        for o in ob:
            for ob in flatten(o):
                yield ob
    when.otherwise() do(ob):
        yield ob

# later:
flatten.when(file) do(f):
    for line in f:
        yield line


###################
# Sort-of a "with(lock)" block, but running the body in a thread.
pool.run_with(my_lock) do:
    # lock is held here
    work()
other_work() # in parallel

###################

Well, that should give you the general idea.

Some final random points:
* As proposed, blocks are a new kind of beast in Python - an indented
block that is part of an expression, rather than being a statement. I
think that by constraining a block to be the last thing in an
expression, this works out OK, but I may be missing something
* Since writing this, I have started to lean towards "def" as the
keyword, rather than "do". I used "do" in the examples mostly to make
them look like Ruby, but I suspect that making Python "look like Ruby"
is not a design goal for most people :)
* Terminology: "block"? "def expression"? "anonymous def"? "lambda++"?
* Idea: Could we relax the rules for where a block is allowed,
removing the constraint that it follows a call or indexing expression?
Then we could do, for example, this:
  def f():
    return def(args):
        pass
* Would a proliferation of nameless functions be a nightmare for
debugging?


Implementation is here: http://bitbucket.org/grammati/python-trunk/
Patch is here: http://bugs.python.org/issue6469
To be honest, I'm not entirely sure if the patch is in the right
format - it comes from hg, not svn.


Chris Perkins