[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

[Darren Dale]

On Fri, Sep 4, 2009 at 3:30 PM, Fernando Perez<fperez.net at gmail.com> wrote:
> On Fri, Sep 4, 2009 at 12:17 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>> Your intent was clear, but the implementation still leaves me
>> wondering what is gained by using the @decorator syntax. Maybe I have
>> missed something, please bear with me.
>>
>> With @decorator, you are still passing a function to another function
>> to modify its execution, its just a different syntax that achieves the
>> same result, isn't it? For example, using your for_each:
>> for_each(range(count))(loop) yields the same result without having to
>> redefine loop each time you call loop_deco.
>
> I think the only point to take home is that these decorators *execute*
> the decorated function right away, and you can ignore it.  Let's look
> only at the loop body:
>
>    for i in range(count):
>        results[i] = do_work(data, i)
>
> What this simple pattern allows us to do is to (say) profile this
> chunk of code right where it is, without having to move it out to
> another separate function (where you'd need to create and pass
> arguments to provide local variables for example),

Ok, I didn't appreciate that you defined func where you did in order
to get access to those local variables. And while I prefer the
decorator syntax, it is still identical to "def block(): ...;
profiled(block)", right?

> by doing two things:
>
> 1. Prepend two lines of code:
>
>    @profiled
>    def block():
>
> 2. Indent by one level the code you wanted to profile, leaving you with:
>
>    @profiled
>    def block():
>        for i in range(count):
>            results[i] = do_work(data, i)
>
> That's it.  All local variables remain available, you don't need to
> pass arguments around, you never actually call block() yourself, nada.

Ok, or:

   def block():
       for i in range(count):
           results[i] = do_work(data, i)
   profiled(block)

>  For example, if you wanted the profiling to happen one level deeper
> instead, you can do this instead:
>
>    for i in range(count):
>        @profiled
>        def block():
>            results[i] = do_work(data, i)
>
> You've just swapped indentation levels, you didn't need to worry about
> passing variables around, and now the profiling happens per-call of
> the loop rather than at the top. This may be a silly example, but it
> just tries to show how the point of this little exercise is just to
> switch scoped execution control with the least intrusive amount of
> work possible.


   for i in range(count):
       def block():
           results[i] = do_work(data, i)
       profiled(block)

> I admit though: this is so trivial, and already in the language, that
> maybe I am indeed happy about nothing :)  So I really appreciate the
> feedback, especially critical one!  If this is pointless, might as
> well kill it before wasting anyone's time with it, so please *do* be
> critical of this.

One other potential problem that just occurred to me: it is a
non-standard use of the decorator syntax. When I first saw it I
thought "wait, where does he call loop?" It happened inside the
decorator, but the standard use is to rebind the returned value to
loop (or block, or whatever). PEP318 doesn't mention this pattern you
have implemented, and I didn't see it at
http://wiki.python.org/moin/PythonDecoratorLibrary either (I did not
do an exhaustive search). In this case, since the decorator syntax has
a very well established purpose, I think the "profiled(block)"
alternative is much more obvious.

Darren