A friendlier, sugarier lambda -- a proposal for Ruby-like blocks in python

[James Stroud]

brenocon at gmail.com wrote:
> Hi all --
> 
> Compared to the Python I know and love, Ruby isn't quite the same.
> However, it has at least one terrific feature: "blocks".  Whereas in
> Python a
> "block" is just several lines of locally-scoped-together code, in Ruby
> a
> "block" defines a closure (anonymous function).  To avoid confusion
> let's call
> them Ruby block-closures.  I see them as just a syntax for defining
> closures
> and passing them into method calls.  I think something analogous could
> be added
> to Python in a very simple manner that would make closures much more
> readable
> and usable, and nail some use cases nicely.
> 
> To define a new closure and pass it into a function call, there are two
> current
> methods: inline 'def' and 'lambda'.  Consider the following Twisted-ish
> code:
> 
>     deferred = fetchPage('http://python.org')
>     def _showResponse(response)
>         print "fancy formatting: %s" % response.text
>     deferred.addCallback(_showResponse)
> 
> Lots of Twisted code has to be written backwards like this.
> Theoretically, it
> might be nice to use lambda right in the addCallback() call, like:
> 
>     deferred.addCallback(lambda r:  print("fancy formatting %s"
> %r.text) )
> 
> But this is awkward since the lambda is constrained to be one line; you
> can't
> come back later and add much to the callback's code.  Furthermore, this
> example
> isn't even legal, because 'print' isn't a function, but a statement --
> lambda
> is further constrained to only contain an expression.
> 
> Many have complained about this crippled-ness of lambda, but it
> actually makes
> some sense.  Since Python uses colons and indentation to define blocks
> of code,
> it would be awkward to close a multiline lambda.  The best I could
> think of
> would look like
> 
>     deferred.addCallback(lambda r:
>         print("fancy formatting %s" % r.text)
>     )
> 
>     ^
>     |
> 
> That trailing paranthesis is WAY un-Pythonic.  We don't close code
> blocks like
> that!  And in general, declaring big multiline anonymous functions in
> the
> middle of a list of normal variable arguments is weird -- it just
> doesn't fit.
> It's perfectly legal to pass in 4 closures, interspersed with number
> and string
> arguments.  Imagine defining all of those inline with 'lambda'
> expressions!
> And what about nesting?  And then there's the term "lambda", while a
> great
> homage to Lisp and computation theory, just isn't the friendliest
> programming
> vocab term.
> 
> (from my limited understanding,) Ruby block-closures assume a specific
> use
> case: You want to pass exactly one multiline, defined-right-there
> closure to a
> method when calling it.  Therefore, you should get to define the
> closure
> *immediately following* the method call.  I suggest a Python version
> with a
> keyword 'using' (or 'with'?) that takes the block of code as a closure,
> and
> passes it to the method call as the *last argument*.  The above example
> becomes:
> 
>     deferred.addCallback() using response:
>         print "fancy formatting %s" % response.text
> 
> and in general, the following two code snippets are equivalent:
> 
> 
>     def _f(x,y):
>         [do stuff with x and y]
>     function_with_callback(a,b,c, _f)
> 
>     function_with_callback(a,b,c) using x,y:
>         [do stuff with x and y]
>     next_statement()
> 
> ... where function_with_callback wants a 2-arg function as its last
> argument.
> It gets to call _f, or equivalently the defined-right-there
> closure/ruby-block,
> however it wants to -- wait for an I/O operation to finish, whatever.
> I'm not
> so hot about the fact that it looks like addCallback() should be
> completed
> before the 'using' keyword kicks in, but this is the simplest I could
> think of.
> 
> This syntax does not let you define a new function and store it as a
> local
> variable.  Python already has inline 'def' for that (that is, you can
> do a
> 'def' in any block of code you want, and it stays local to that scope.)
>  It
> does not, strictly speaking, let you do anything new -- as Guido has
> stated,
> you could ditch lambda and achieve the equivalent by declaring the
> little
> callback function as an inline 'def', like in the first deferred
> example here.
> 
> This only optimizes for the case of defining a closure only for the
> purpose of
> passing it in as an argument to a method.  However, this must be the
> only use
> for anonymous functions in Python, since we already have inline 'def'.
> I've
> always felt that passing in a lambda in lisp and python, or the
> equivalent
> anonymous function(x,y) {} in javascript, or anonymous classes in java,
> was
> always awkward.  You have to think about defining a new anonymous
> function
> *within* your method call and such, and then you have to close
> parantheses
> after it -- it's just weird, I dunno.
> 
> This proposal could also handle some of the use cases mentioned in PEP
> 310 and
> 340.  If you design your callback-taking functions to have only one
> callback
> and have it as the last argument, you can trivially write lock
> acquition (omit
> 'using' for a no-arg block-closure):
> 
>     def protect(lock, f):
>         lock.acquire()
>         f()
>         lock.release()
> 
>     protect(myLock):
>         [do stuff that needs myLock to be acquired]
> 
> Of course, the definition of protect() might have try/finally wrapped
> around
> the f() call.  (Interestingly, this starts looking like a way to define
> new
> control-like structures.  I haven't thought through the implications.)
> 
> ActiveRecord, Rails' object-relational mapper, does almost exactly this
> for
> database transactions, and I have found it quite nice:
> 
>     # User is a sqlobject/sqlalchemy/django/whatever ORM class;
>     # User.transaction is a class method executing its passed-in
> closure within
>     #  the user table's START TRANSACTION and STOP TRANSACTION.
> 
>     user1, user2 = getTwoUsers()
>     User.transaction() using user1, user2:
>         someRaceConditionProneStuff(user1, user2)
>         moreRaceConditionProneStuff(user1, user2)
> 
> There might be some sort of overlap with PEP 343 and the 'with'
> statement, but
> I'm not sure exactly.  Sorry I'm late to the game and commenting on
> last year's
> PEP's, but I've only started reading them.  Note that PEP's 343 and 340
> are
> very focused on resource management -- but I think that letting one
> define code
> blocks as closures could make resource handling routines be easily
> written in
> Python.  Furthermore, tons more stuff -- like Deferreds and such --
> could be
> added.  (Ruby uses block-closures to do really basic constructs such as
> foreach
> iteration.  Python does a fine job with "for x in L" and list and
> generator
> comprehensions... enough so that map/lambda is obsolete!  I'm just
> trying to
> see if there are use cases in Python for block-closures.)
> 
> I've been trying to search for similar proposals but have come up dry.
> Anything like this out there?  I hear Ruby blocks are heavily inspired
> by
> Smalltalk; anyone know more?
> 
> Is it feasible to assume the primary use of closures is as part of an
> argument
> list, and such argument lists will want only one argument that is a
> closure?
> Does doing so avoid any big annoyances of functional programming?
> 
> Is this completely, totally incompatible with the current state of
> Python and
> previous deliberations :) ?  e.g. I haven't thought much about how this
> would
> interact with yield and generators.
> 
> But really, I'm just idly curious -- does anyone think this might be
> useful?
> 
> 
> Take care,
> Brendan
> 

http://mail.python.org/pipermail/python-list/2004-April/215805.html