[Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Peter O'Connor]

* correction to example:

moving_average_gen = (average:= moving_average_step(average, x,
decay=decay) for
x in signal from average=initial)

On Thu, Apr 12, 2018 at 3:37 PM, Peter O'Connor <peter.ed.oconnor at gmail.com>
wrote:

> On Wed, Apr 11, 2018 at 10:50 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> In particular, I'm happiest with the named moving_average() function,
>> which may reflect to some extent my lack of familiarity with the
>> subject area. I don't *care* how it's implemented internally - an
>> explicit loop is fine with me, but if a domain expert wants to be
>> clever and use something more complex, I don't need to know. An often
>> missed disadvantage of one-liners is that they get put inline, meaning
>> that people looking for a higher level overview of what the code does
>> get confronted with all the gory details.
>
>
> I'm all in favour of hiding things away into functions - I just think
> those functions should be as basic as possible, without implicit
> assumptions about how they will be used.  Let me give an example:
>
> ----
>
> Lets look at your preferred method (A):
>
>     def moving_average(signal_iterable, decay, initial=0):
>         last_average = initial
>         for x in signal_iterable:
>             last_average = (1-decay)*last_average + decay*x
>             yield last_average
>
>     moving_average_gen = moving_average(signal, decay=decay,
> initial=initial)
>
> And compare it with (B), which would require the proposed syntax:
>
>     def moving_average_step(last_average, x, decay):
>         return (1-decay)*last_average + decay*x
>
>     moving_average_gen = (average:= moving_average_step(average, x,
> decay=decay) for x in signal from x=initial)
>
> -----
>
> Now, suppose we want to change things so that the "decay" changes with
> every step.
>
> The moving_average function (A) now has to be changed, because what we
> once thought would be a fixed parameter is now a variable that changes
> between calls.  Our options are:
> - Make "decay" another iterable (in which case other functions calling
> "moving_average" need to be changed).
> - Leave an option for "decay" to be a float which gets transformed to an
> iterable with "decay_iter = (decay for _ in itertools.count(0)) if
> isinstance(decay, (int, float)) else decay".  (awkward because 95% of
> usages don't need this.  If you do this for more parameters you suddenly
> have this weird implementation with iterators everywhere even though in
> most cases they're not needed).
> - Factor out the "pure"  "moving_average_step" from "moving_average", and
> create a new "moving_average_with_dynamic_decay" wrapper (but now we have
> to maintain two wrappers - with the duplicated arguments - which starts to
> require a lot of maintenance when you're passing down several parameters
> (or you can use the dreaded **kwargs).
>
> With approach (B) on the other hand, "moving_average_step" and all the
> functions calling it, can stay the same: we just change the way we call it
> in this instance to:
>
>     moving_average_gen = (average:= moving_average_step(average, x,
> decay=decay) for x, decay in zip(signal, decay_schedule) from x=initial)
>
> ----
>
> Now lets imagine this were a more complex function with 10 parameters.  I
> see these kind of examples a lot in machine-learning and robotics programs,
> where you'll have parameters like "learning rate", "regularization",
> "minibatch_size", "maximum_speed", "height_of_camera" which might initially
> be considered initialization parameters, but then later it turns out they
> need to be changed dynamically.
>
> This is why I think the "(y:=f(y, x) for x in xs from y=initial)" syntax
> can lead to cleaner, more maintainable code.
>
>
>
> On Wed, Apr 11, 2018 at 10:50 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> On 11 April 2018 at 15:37, Peter O'Connor <peter.ed.oconnor at gmail.com>
>> wrote:
>>
>> > If people are happy with these solutions and still see no need for the
>> > initialization syntax, we can stop this, but as I see it there is a
>> "hole"
>> > in the language that needs to be filled.
>>
>> Personally, I'm happy with those solutions and see no need for the
>> initialisation syntax.
>>
>> In particular, I'm happiest with the named moving_average() function,
>> which may reflect to some extent my lack of familiarity with the
>> subject area. I don't *care* how it's implemented internally - an
>> explicit loop is fine with me, but if a domain expert wants to be
>> clever and use something more complex, I don't need to know. An often
>> missed disadvantage of one-liners is that they get put inline, meaning
>> that people looking for a higher level overview of what the code does
>> get confronted with all the gory details.
>>
>> Paul
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180412/a08b34fa/attachment-0001.html>