[Python-ideas] Ruby-style Blocks in Python Idea [with examples]

[Guido van Rossum]

On Mon, Mar 9, 2009 at 11:47 AM, tav <tav at espians.com> wrote:
> Here's another stab at making a case for it. I'll avoid referring to
> Ruby this time round -- I was merely using it as an example of where
> this approach has been successful. Believe me, there's no Ruby-envy
> here ;p
>
> The motivation:
>
> 1. Having to name a one-off function adds additional cognitive
> overload to a developer. It doesn't make the code any cleaner and by
> taking away the burden, we'd have happier developers and cleaner code.

I showed an example using "def _(...)".

> 2. This approach is more descriptive and in line with the code flow.
> With blocks the first line says "I'm about to define a function for
> use with X" instead of the existing way which says "I'm defining a
> function. Now I'm using that function with X."

Marginal. The decorator name could clarify this too.

> 3. DSLs -- whether we like them or not, they are in mainstream use.
> Python already has beautiful syntax. We should be leveraging that for
> DSLs instead of forcing framework developers to create their own
> ugly/buggy mini-DSLs. This will enable that.

This is quite the non-sequitur. Given that what you propose is
trivially done using a decorator, what's ugly/buggy about the existing
approach?

> The proposed syntax:
>
>  using EXPR do PARAM_LIST:
>    FUNCTION_BODY
>
> Examples:
>
> # Django/App Engine Query
>
> Frameworks like Django or App Engine define DSLs to enable easy
> querying of datastores by users. Wouldn't it better if this could be
> done in pure Python syntax?
>
> Compare the current Django:
>
>  q = Entry.objects.filter(headline__startswith="What").filter(pub_date__lte=datetime.now())
>
> with a hypothetical:
>
>  using Entry.filter do (entry):
>      if entry.headline.startswith('What') and entry.pub_date <= datetime.now():
>          return entry

Hmm... where does the 'return' return to? The "current Django" has an
assignment to q. How do you set q in this example?

> Wouldn't the latter be easier for a developer to read/maintain?

But it's not the same. If they had wanted you to be able to write that
they could easily have provided an iterator (and actually of course
they do -- it's the iterator over all records). But the (admittedly
awkward) current syntax *executes the query in the database engine*.
There's no way (without proposing a lot of other changes to Python
anyway) to translate the Python code in the body of your example to
SQL, because Python (the language, anyway -- not all implementations
support access to the bytecode) doesn't let you recover the source
code of a block at run time.

> Let's compare this App Engine:
>
>  composer = "Lennon, John"
>  query = GqlQuery("SELECT * FROM Song WHERE composer = :1", composer)
>
> with:
>
>  composer = "Lennon, John"
>  using Song.query do (item):
>      if item.composer == composer:
>          return item
>
> Again, being able to do it in Python syntax will save developers the
> hassles of having to learn non-Python DSLs.

Again, it's broken the same way.

> # Event-driven Programming
>
> Right now, event-driven programming like it's done in Twisted is
> rather painful for many developers. It's filled with callbacks and the
> order in which code is written is completely inverted as far as the
> average developer is concerned.
>
> Let's take Eventlet -- a nice coroutines-based networking library in
> Python. Their example webcrawler.py currently does:
>
>  urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
>          "http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
>
>  def fetch(url):
>      print "%s fetching %s" % (time.asctime(), url)
>      httpc.get(url)
>      print "%s fetched %s" % (time.asctime(), url)
>
>  pool = coros.CoroutinePool(max_size=4)
>  waiters = []
>
>  for url in urls:
>      waiters.append(pool.execute(fetch, url))
>
> Wouldn't it be nicer to do this instead:
>
>  pool = coros.CoroutinePool(max_size=4)
>
>  for url in urls:
>      using pool.execute do:
>          print "%s fetching %s" % (time.asctime(), url)
>          httpc.get(url)
>          print "%s fetched %s" % (time.asctime(), url)
>
> I'd argue that it is -- but then I have bias =)

It's also broken -- unless you also have some kind of alternative
semantics in mind that does *not* map to existing Python functions and
scopes, all callbacks will reference the last url in the list. Compare
this classic stumbling block:

addN = [(lambda x: x+i) for i in range(10)]
add1 = addN[1]
print add1(10)  # prints 19

> # SCons
>
> SCons is a make-esque build tool. In the SConstruct (makefile) for
> Google Chrome, we find:
>
>  def WantSystemLib(env, lib):
>      if lib not in env['all_system_libs']:
>          env['all_system_libs'].append(lib)
>      return (lib in env['req_system_libs'])
>  root_env.AddMethod(WantSystemLib, "WantSystemLib")
>
> Which we could hypothetically do as:
>
>  with root_env.WantSystemLib do (env, lib):
s/with/using/
>      if lib not in env['all_system_libs']:
>          env['all_system_libs'].append(lib)
>      return (lib in env['req_system_libs'])

That's just a matter of API design. They could easily have provided a
decorator to register the callback. The name of the decorated function
could even serve as the key.

> As someone who's used both make and SCons, I found SCons terribly
> verbose and painful to use. By using the proposed do statement, SCons
> could be made extremely pleasant!

Quite the exaggeration.

> # Webapp Configuration
>
> Configuration in web applications is generally a real pain:
>
>  application = webapp([('/profile', ProfileHandler), ('/', MainHandler)],
>                                   debug=True)
>  run(application)
>
> Compare to:
>
>  using webapp.runner do (config, routes):
>      routes['/profiles'] = ProfileHandler
>      routes['/'] = MainHandler
>      config.debug = True
>
> I think the latter is more readable and maintainable.

Again, there's no need to add new syntax if you wanted to make this
API easier on the eyes.

> Please let me know if more examples would help...

Well, examples that (a) aren't broken and (b) aren't trivially written
using decorators might help...

> I really do believe that a block syntax would make developers more
> productive and lead to cleaner code.

I got that. :-)

> --
> love, tav
>
> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
> http://tav.espians.com | http://twitter.com/tav | skype:tavespian
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)