[Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Peter O'Connor]

On Wed, Apr 11, 2018 at 10:50 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> In particular, I'm happiest with the named moving_average() function,
> which may reflect to some extent my lack of familiarity with the
> subject area. I don't *care* how it's implemented internally - an
> explicit loop is fine with me, but if a domain expert wants to be
> clever and use something more complex, I don't need to know. An often
> missed disadvantage of one-liners is that they get put inline, meaning
> that people looking for a higher level overview of what the code does
> get confronted with all the gory details.


I'm all in favour of hiding things away into functions - I just think those
functions should be as basic as possible, without implicit assumptions
about how they will be used.  Let me give an example:

----

Lets look at your preferred method (A):

    def moving_average(signal_iterable, decay, initial=0):
        last_average = initial
        for x in signal_iterable:
            last_average = (1-decay)*last_average + decay*x
            yield last_average

    moving_average_gen = moving_average(signal, decay=decay,
initial=initial)

And compare it with (B), which would require the proposed syntax:

    def moving_average_step(last_average, x, decay):
        return (1-decay)*last_average + decay*x

    moving_average_gen = (average:= moving_average_step(average, x,
decay=decay) for x in signal from x=initial)

-----

Now, suppose we want to change things so that the "decay" changes with
every step.

The moving_average function (A) now has to be changed, because what we once
thought would be a fixed parameter is now a variable that changes between
calls.  Our options are:
- Make "decay" another iterable (in which case other functions calling
"moving_average" need to be changed).
- Leave an option for "decay" to be a float which gets transformed to an
iterable with "decay_iter = (decay for _ in itertools.count(0)) if
isinstance(decay, (int, float)) else decay".  (awkward because 95% of
usages don't need this.  If you do this for more parameters you suddenly
have this weird implementation with iterators everywhere even though in
most cases they're not needed).
- Factor out the "pure"  "moving_average_step" from "moving_average", and
create a new "moving_average_with_dynamic_decay" wrapper (but now we have
to maintain two wrappers - with the duplicated arguments - which starts to
require a lot of maintenance when you're passing down several parameters
(or you can use the dreaded **kwargs).

With approach (B) on the other hand, "moving_average_step" and all the
functions calling it, can stay the same: we just change the way we call it
in this instance to:

    moving_average_gen = (average:= moving_average_step(average, x,
decay=decay) for x, decay in zip(signal, decay_schedule) from x=initial)

----

Now lets imagine this were a more complex function with 10 parameters.  I
see these kind of examples a lot in machine-learning and robotics programs,
where you'll have parameters like "learning rate", "regularization",
"minibatch_size", "maximum_speed", "height_of_camera" which might initially
be considered initialization parameters, but then later it turns out they
need to be changed dynamically.

This is why I think the "(y:=f(y, x) for x in xs from y=initial)" syntax
can lead to cleaner, more maintainable code.



On Wed, Apr 11, 2018 at 10:50 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 11 April 2018 at 15:37, Peter O'Connor <peter.ed.oconnor at gmail.com>
> wrote:
>
> > If people are happy with these solutions and still see no need for the
> > initialization syntax, we can stop this, but as I see it there is a
> "hole"
> > in the language that needs to be filled.
>
> Personally, I'm happy with those solutions and see no need for the
> initialisation syntax.
>
> In particular, I'm happiest with the named moving_average() function,
> which may reflect to some extent my lack of familiarity with the
> subject area. I don't *care* how it's implemented internally - an
> explicit loop is fine with me, but if a domain expert wants to be
> clever and use something more complex, I don't need to know. An often
> missed disadvantage of one-liners is that they get put inline, meaning
> that people looking for a higher level overview of what the code does
> get confronted with all the gory details.
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180412/4d4db33a/attachment-0001.html>