<div dir="ltr"><div><div><div><span style="font-size:12.8px">The idea is to let generator expressions and list/set comprehensions have a clean syntax to access its last output. That would allow them to be an alternative syntax to the scan higher-order function [1] (today implemented in the itertools.accumula</span>te function), which leads to an alternative way to write a fold/reduce. It would be nice to have something like:<br><br><span style="font-family:monospace,monospace">>>> last(abs(prev - x) for x in [3, 4, 5] from prev = 2)<br>2</span><br><br>instead of a reduce:<br><br></div><div><span style="font-family:monospace,monospace">>>> from functools import reduce<br></span></div><div><span style="font-family:monospace,monospace">>>> reduce(lambda prev, x: abs(prev - x), [3, 4, 5], 2)<br>2</span><br><br></div><div>or an imperative approach:<br><br><span style="font-family:monospace,monospace"></span></div><div><span style="font-family:monospace,monospace">>>> prev = 2<br></span></div><div><span style="font-family:monospace,monospace">>>> for x in [3, 4, 5]:<br></span></div><div><span style="font-family:monospace,monospace">... prev = abs(prev - x)<br></span></div><div><span style="font-family:monospace,monospace">>>> prev<br></span></div><div><span style="font-family:monospace,monospace">2<br></span><br>or getting the last from accumulate:<br><br></div><div><span style="font-family:monospace,monospace">>>> from itertools import accumulate<br></span></div><div><span style="font-family:monospace,monospace">>>> list(accumulate(</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace">[2, 3, 4, 5], </span></span>lambda prev, x: abs(prev - x)</span>))[-1]<br>2<br></span></div><span style="font-family:monospace,monospace"></span><div><br>or...<br><br><div><span style="font-family:monospace,monospace">>>> [prev for prev in [2]<br>... for x in [3, 4, 5]<br>... for prev in [abs(prev - x)]<br>... ][-1]<br>2</span><span style="font-family:monospace,monospace"></span><br><span style="font-family:monospace,monospace"></span></div><br>Actually, I already wrote a solution for something similar to that: PyScanPrev [2]. I'm using bytecode manipulation to modify the generator expression and set/list comprehensions seman<span style="font-size:12.8px"><span style="font-size:12.8px">tics to create a "scan"</span>, but it has the limitation of using only code with a valid syntax as the input, so I can't use "from" inside a generator expression / list comprehension. The solution was to put the first output into the iterable and define the "prev" name elsewhere:<br></span></div><div><br><span style="font-family:monospace,monospace">>>> last(abs(prev - x) for x in [2, 3, 4, 5])<br>2</span><br><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">That line works with PyScanPrev (on Python 3.4 and 3.5) when defined in a function with a <span style="font-family:monospace,monospace">@enable_scan("prev")</span> decorator. That was enough to create a "test suite" of doctest-based examples that shows several scan use cases [2].<br></span></div><div><span style="font-size:12.8px"><br>This discussion started in a Brazilian list when someone asked how she could solve a simple uppercase/lowercase problem [3]. The goal was to alternate the upper/lower case of a string while neglecting the chars that doesn't apply (i.e., to "keep the state" when the char isn't a letter). </span>A<span style="font-size:12.8px">fter the discussion, I wrote the PyScanPrev package, and recently I've added this historical "alternate" function as the </span><span style="font-size:12.8px"><span style="font-size:12.8px"><span style="font-size:12.8px">"conditional toggling" example [4].</span></span><br></span><br></div>Then I ask, can Python include that "scan" access to the last output in its list/set/dict comprehension and generator expression syntax? There are several possible applications for the scan itself as well as for the fold/reduce (signal processing, control theory, physics, economics, etc.), some of them I included as PyScanPrev examples. Some friends (people who like control engineering and/or signal processing) liked the "State-space model" example, where I included a "leaking bucket-spring-damper" simulation using the scan-enabled generator expressions [5].<br><br>About the syntax, there are several ideas on how that can be written. Given a "prev" identifier, a "target" identifier, an input "iterable" and an optional "start" value (and perhaps an optional "echo_start", which I assume True by default), some of them are:<br><br></div><div></div><span style="font-family:monospace,monospace">[func(prev, target) for target in iterable from prev = start]<br></span><div><span style="font-family:monospace,monospace">[func(prev, target) for target in iterable] -> prev = start<br><span class="gmail-"></span>[func(prev, target) for target in iterable] -> prev as start<br><span class="gmail-"></span>[func(prev, target) for target in iterable] from prev = start<br><span class="gmail-"></span><span class="gmail-"></span>[func(prev, target) for target in iterable] from prev as start<br><span class="gmail-"></span><span class="gmail-"></span><span class="gmail-"></span>[func(prev, target) for target in iterable] with prev as start<br></span></div><div><span style="font-family:monospace,monospace">prev = start -> [func(prev, target) for target in iterable]<br></span></div><div><span style="font-family:monospace,monospace"><span class="gmail-"></span></span><div><span style="font-family:monospace,monospace">prev(start) -> [func(prev, target) for target in iterable]<br></span><div><span style="font-family:monospace,monospace">[func(prev, target) for prev -> target in start -> iterable]<br></span></div><div><span style="font-family:monospace,monospace">[prev = start -> func(prev, target) for target in iterable]<br></span></div></div><span style="font-family:monospace,monospace"><br></span><div><span style="font-family:monospace,monospace"># With `</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace">`</span>start</span><span style="font-family:monospace,monospace"><span style="font-family:monospace,monospace">`</span>` being the first value of the iterable, i.e.,<br># iterable = prepend(start, data)<br></span></div><span style="font-family:monospace,monospace">[func(prev, target) for target in iterable from prev]<br>[func(prev, target) for target in iterable] -> prev<br>[func(prev, target) for target in iterable] from prev<br>prev -> [func(prev, target) for target in iterable]</span><br></div><br></div><div>Before writing PyScanPrev, in [6] (Brazilian Portuguese) I used stackfull [7] to implement that idea, an accumulator example using that library is:<br><br></div><span style="font-family:monospace,monospace">>>> from stackfull import push, pop, stack<br>>>> [push(pop() + el if stack() else el) for el in range(5)]<br>[0, 1, 3, 6, 10]<br></span></div><span style="font-family:monospace,monospace">>>> list(itertools.accumulate(range(5)))</span><br><span style="font-family:monospace,monospace"></span><div><span style="font-family:monospace,monospace">[0, 1, 3, 6, 10]</span><br><br>There are more I can say (e.g. the pyscanprev.scan function has a "start" value and an "echo_start" keyword argument, resources I missed in itertools.accumulate) but the links below already have a lot of information.<br><div><div><br><span style="font-size:12.8px">[1] <a href="https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function" target="_blank">https://en.wikipedia.org/wiki/<wbr>Prefix_sum#Scan_higher_order_f<wbr>unction</a></span></div><div><div><span style="font-size:12.8px">[2] <a href="https://pypi.python.org/pypi/pyscanprev">https://pypi.python.org/pypi/pyscanprev</a></span><span style="font-size:12.8px"></span></div><div><span style="font-size:12.8px"></span></div><div><span style="font-size:12.8px">[3] <a href="https://groups.google.com/forum/#%21topic/grupy-sp/wTIj6G5_5S0" target="_blank">https://groups.google.com/foru<wbr>m/#!topic/grupy-sp/wTIj6G5_5S0</a><br>[4] <a href="https://github.com/danilobellini/pyscanprev/blob/v0.1.0/examples/conditional-toggling.rst">https://github.com/danilobellini/pyscanprev/blob/v0.1.0/examples/conditional-toggling.rst</a><br>[5] <a href="https://github.com/danilobellini/pyscanprev/blob/v0.1.0/examples/state-space.rst">https://github.com/danilobellini/pyscanprev/blob/v0.1.0/examples/state-space.rst</a><br></span>[6] <a href="https://groups.google.com/forum/#%21topic/grupy-sp/UZp-lVSWK1s" target="_blank">https://groups.google.com/foru<wbr>m/#!topic/grupy-sp/UZp-lVSWK1s</a><br><span style="font-size:12.8px">[7] <a href="https://pypi.python.org/pypi/stackfull">https://pypi.python.org/pypi/stackfull</a><br><br></span></div>-- <br></div><div>Danilo J. S. Bellini<br>---------------<br>"<i>It is not our business to set up prohibitions, but to arrive at conventions.</i>" (R. Carnap)<br></div>
</div></div></div>