[Python-ideas] Pass a function as the argument "step" of range()

[Pierre Quentel]

2015-07-03 13:23 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:

> On 3 July 2015 at 06:20, Pierre Quentel <pierre.quentel at gmail.com> wrote:
> > @Steven, Mark
> > The definition of range() in Python docs says :
> >
> > Python 2.7 : "This is a versatile function to create lists containing
> > arithmetic progressions. It is most often used in for loops."
> >
> > Python 3.4 : "The range type represents an immutable sequence of numbers
> and
> > is commonly used for looping a specific number of times in for loops."
>
> Pierre, I *wrote* the Python 3 range docs. I know what they say.
> Functionality for generating an arbitrary numeric series isn't going
> into range().
>
> Now, it may be that there's value in having a way to neatly express a
> potentially infinite mathematical series, and further value in having
> a way to terminate iteration of that series based on the values it
> produces.
>
> The key question you need to ask yourself is whether or not you can
> come up with a proposal that is easier to read than writing an
> appropriately named custom iterator for whatever iteration problem you
> need to solve, or using a generator expression with
> itertools.takewhile and itertools.count:
>
>     from itertools import takewhile, count
>     for i in takewhile((lambda i: i < N), (2**x for x in count())):
>         ...
>
> Outside obfuscated code contests, there aren't any prizes for
> expressing an idea in the fewest characters possible, but there are
> plenty of rewards to be found in expressing ideas in such a way that
> future maintainers can understand not only what the code actually
> does, but what it was intended to do, and that the computer can also
> execute at an acceptable speed.
>
> Assuming you're able to come up with such a proposal, the second
> question would then be whether that solution even belongs in the
> standard library, let alone in the builtins. What are the real world
> problems that the construct solves that itertools doesn't already
> cover? Making it easier to translate homework assignments written to
> explore features of other programming languages rather than features
> of Python doesn't count.
>
> > Both stress that range is most often used in a for loop (it doesn't
> "happens
> > to sometimes be used" in for loops, and is rarely used for membership
> > testing). Python 2.7 limited its definition to arithmetic progressions,
> but
> > Python 3.4 has a more general definition (an immutable sequence of
> numbers).
> > I really don't think that the proposal would change the general idea
> behind
> > range : a suite of integers, where each item is built from the previous
> > following a specific pattern, and stopping when a "stop" value is
> reached.
>
> There isn't a "general idea" behind Python 3's range type, there's a
> precise, formal definition.
>
> For starters, the contents are defined to meet a specific formula:
> ===================
> For a positive step, the contents of a range r are determined by the
> formula r[i] = start + step*i where i >= 0 and r[i] < stop.
>
> For a negative step, the contents of the range are still determined by
> the formula r[i] = start + step*i, but the constraints are i >= 0 and
> r[i] > stop.
> ===================
>
> If you're tempted to respond "we can change the formula to use an
> arbitrary element value calculation algorithm", we make some *very*
> specific performance and behavioural promises for range objects, like:
>
> ===================
> Range objects implement the collections.abc.Sequence ABC, and provide
> features such as containment tests, element index lookup, slicing and
> support for negative indices
> ===================
> Testing range objects for equality with == and != compares them as
> sequences. That is, two range objects are considered equal if they
> represent the same sequence of values. (Note that two range objects
> that compare equal might have different start, stop and step
> attributes, for example range(0) == range(2, 1, 3) or range(0, 3, 2)
> == range(0, 4, 2).)
> ===================
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>

Nick,

Thanks for taking the time to explain. I am conscious that my proposal is
not well received (understatement), and I respect the opinion of all those
who expressed concerns with it. For me Terry's objection is the most
serious : with a function instead of a fixed increment, many current
methods of range() can't be implemented, or with a serious performance
penalty.

This pretty much kills the discussion. Nevertheless I will try to answer
your questions.

The proposal is (was) to extend the incrementation algorithm used to
produce items from range() : from an addition of a fixed step to the last
item, to an arbitrary function on the last item.

The best I can do is rewriting the first lines of the document of range() :

###
class range(stop)
class range(start, stop[, step])

The arguments start and stop to the range constructor must be integers
(either built-in int or any object that implements the __index__ special
method). If the start argument is omitted, it defaults to 0.

The step argument can be an integer of a function. If it is omitted, it
defaults to 1. If step is zero, ValueError is raised.

If step is a positive integer, the contents of a range r are determined by
the formula r[i] = start + step*i where i >= 0 and r[i] < stop.

If step is a negative integer, the contents of the range are still
determined by the formula r[i] = start + step*i, but the constraints are i
>= 0 and r[i] > stop.

If step is a function, the contents of the range is determined by the
formulas r[0] = start, r[i] = step(r[i-1]) where i >= 1. If stop > step,
the iteration stops when r[i] >= stop ; if stop < start, when r[i] <= stop.
###

The advantage over specific generators or functions in itertools is the
generality of the construct. For the example with the sequence of powers of
2, I find that

for i in range(1, N, lambda x:x*2):
    ...

is more readable than :

from itertools import takewhile, count
for i in takewhile((lambda i: i < N), (2**x for x in count())):
    ...

It is not because it is shorter : I hate obscure one-liners as much as
anyone. It is for two main reasons :
- it makes it clear that we start at 1, we stop at N, and the
incrementation is done by multiplying the previous item by 2.
- the second form requires mastering the functions in itertools, which is
not the case of all Python developers - after all, itertools is a module,
its functions are not built-in. Even those who do hesitate between count()
and accumulate().

Moreover, the construct applies to any incrementing function ; with
itertools, you need to translate the function using the appropriate
function(s) in the module.

Of course this doesn't solve any problem that can't be solved any other
way. But for that matter, there is nothing comprehensions can do that can't
be done with loops - not that I compare my proposal to comprehensions in
any way, it's just to say that this argument is not an absolute killer.

Once again thank you all for your time.
Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150703/d6dbf483/attachment.html>