Kyle, you sounded so reasonable when you were trashing itertools.accumulate (which I now agree is horrible).  But then you go and support Serhiy's madness:  "smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]" which I agree is clever, but reads more like a riddle than readable code.  

Anyway, I continue to stand by:

    (y:= f(y, x) for x in iter_x from y=initial_y)

And, if that's not offensive enough, to its extension:

    (z, y := f(z, x) -> y for x in iter_x from z=initial_z)

Which carries state "z" forward but only yields "y" at each iteration.  (see proposal: https://github.com/petered/peps/blob/master/pep-9999.rst)

Why am I so obsessed?  Because it will allow you to conveniently replace classes with more clean, concise, functional code.  People who thought they never needed such a construct may suddenly start finding it indispensable once they get used to it.  

How many times have you written something of the form?:

    class StatefulThing(object):
    
        def __init__(self, initial_state, param_1, param_2):
            self._param_1= param_1 
            self._param_2 = param_2 
            self._state = initial_state
    
        def update_and_get_output(self, new_observation):  # (or just __call__)
            self._state = do_some_state_update(self._state, new_observation, self._param_1) 
            output = transform_state_to_output(self._state, self._param_2)
            return output
    
    processor = StatefulThing(initial_state = initial_state, param_1 = 1, param_2 = 4)
    processed_things = [processor.update_and_get_output(x) for x in x_gen]
    
I've done this many times.  Video encoding, robot controllers, neural networks, any iterative machine learning algorithm, and probably lots of things I don't know about - they all tend to have this general form.  

And how many times have I had issues like "Oh no now I want to change param_1 on the fly instead of just setting it on initialization, I guess I have to refactor all usages of this class to pass param_1 into update_and_get_output instead of __init__".  

What if instead I could just write:

    def update_and_get_output(last_state, new_observation, param_1, param_2)
        new_state = do_some_state_update(last_state, new_observation, _param_1) 
        output = transform_state_to_output(last_state, _param_2)
        return new_state, output

    processed_things = [state, output:= update_and_get_output(state, x, param_1=1, param_2=4) -> output for x in observations from state=initial_state] 

Now we have:
- No mutable objects (which cuts of a whole slew of potential bugs and anti-patterns familiar to people who do OOP.) 
- Fewer lines of code
- Looser assumptions on usage and less refactoring. (if I want to now pass in param_1 at each iteration instead of just initialization, I need to make no changes to update_and_get_output).
- No need for state getters/setters, since state is is passed around explicitly.

I realize that calling for changes to syntax is a lot to ask - but I still believe that the main objections to this syntax would also have been raised as objections to the now-ubiquitous list-comprehensions - they seem hostile and alien-looking at first, but very lovable once you get used to them.  




On Sun, Apr 8, 2018 at 1:41 PM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:


On 2018-04-05 21:18, Steven D'Aprano wrote:
> (I don't understand why so many people have such an aversion to writing
> functions and seek to eliminate them from their code.)
>

I think I am one of those people that have an aversion to writing functions!

I hope you do not mind that I attempt to explain my aversion here. I
want to clarify my thoughts on this, and maybe others will find
something useful in this explanation, maybe someone has wise words for
me. I think this is relevant to python-ideas because someone with this
aversion will make different language suggestions than those that don't.

Here is why I have an aversion to writing functions: Every unread
function represents multiple unknowns in the code. Every function adds
to code complexity by mapping an inaccurate name to specific
functionality. 

When I read code, this is what I see:

>    x = you_will_never_guess_how_corner_cases_are_handled(a, b, c)
>    y =
you_dont_know_I_throw_a_BaseException_when_I_do_not_like_your_arguments(j,
k, l)

Not everyone sees code this way: I see people read method calls, make a
number of wild assumptions about how those methods work, AND THEY ARE
CORRECT!  How do they do it!?  It is as if there are some unspoken
convention about how code should work that's opaque to me.

For example before I read the docs on
itertools.accumulate(list_of_length_N, func), here are the unknowns I see:

* Does it return N, or N-1 values?
* How are initial conditions handled?
* Must `func` perform the initialization by accepting just one
parameter, and accumulate with more-than-one parameter?
* If `func` is a binary function, and `accumulate` returns N values,
what's the Nth value?
* if `func` is a non-cummutative binary function, what order are the
arguments passed? 
* Maybe accumulate expects func(*args)?
* Is there a window size? Is it equal to the number of arguments of `func`?

These are not all answered by reading the docs, they are answered by
reading the code. The code tells me the first value is a special case;
the first parameter of `func` is the accumulated `total`; `func` is
applied in order; and an iterator is returned.  Despite all my
questions, notice I missed asking what `accumulate` returns? It is the
unknown unknowns that get me most.

So, `itertools.accumulate` is a kinda-inaccurate name given to a
specific functionality: Not a problem on its own, and even delightfully
useful if I need it often. 

What if I am in a domain where I see `accumulate` only a few times a
year? Or how about a program that uses `accumulate` in only one place?
For me, I must (re)read the `accumulate` source (or run the caller
through the debugger) before I know what the code is doing. In these
cases I advocate for in-lining the function code to remove these
unknowns. Instead of an inaccurate name, there is explicit code. If we
are lucky, that explicit code follows idioms that make the increased
verbosity easier to read.

Consider Serhiy Storchaka's elegant solution, which I reformatted for
readability

> smooth_signal = [
>     average
>     for average in [0]
>     for x in signal
>     for average in [(1-decay)*average + decay*x]
> ]

We see the initial conditions, we see the primary function, we see how
the accumulation happens, we see the number of returned values, and we
see it's a list. It is a compact, easy read, from top to bottom. Yes, we
must know `for x in [y]` is an idiom for assignment, but we can reuse
that knowledge in all our other list comprehensions.  So, in the
specific case of this Reduce-Map thread, I would advocate using the list
comprehension. 

In general, all functions introduce non-trivial code debt: This debt is
worth it if the function is used enough; but, in single-use or rare-use
cases, functions can obfuscate.



Thank you for your time.









_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/