On Tue, Nov 23, 2021 at 1:18 AM Remy <raimi.bkarim@gmail.com> wrote:
Hi, I'd like to revive this thread after what seemed like a decade and see where this might take us 😃
I like this idea that the OP suggested but I'd like to extend this to all iterator objects (range_iterators, list_iterators etc.).
📌Idea Iterables to expose the .__iter__() method in iterable as .iter(). Iterators to implement 'transformational' functions like map, filter, flat_map, 'reductional' functions like reduce, sum, join, and 'evaluate' functions like to_list and to_set.
📌Why? 1. It is readable in a manner that is procedural and dataflow centric. At one glance, it is easy to reason about how our data gets transformed - we start with the subject, our data, then we perform a sequence of transformations. The previous way of doing this: list( filter( lambda x: x%2==0, map( lambda x: x+1, iter([1,2,3])))) incurs a lot of cognitive overload. We could separate them into different lines: it = iter([1,2,3]) it = map(lambda x: x+1, it) it = filter(lambda x: x%2==0, it) list(it) but I would argue that there is a lot of repetition.
Here's the equivalent as a list comprehension, which I think looks better than either of the above: [x + 1 for x in [1,2,3] if x % 2 == 0]
2. There are existing communities which deal with frameworks that focus on data transformations. This includes PySpark and pandas.
3. It is conventional - many modern languages like Rust, Scala, JavaScript and Kotlin have a similar API like this.
4. The existing map, filter APIs are not so commonly used (this is the impression I have). Apparently list comprehension appears more pythonic on StackOverflow posts. This extended iterator API could possibly 'revive' the use of lazy evaluation.
Comprehensions ARE lazily evaluated. If you use to_list at the end (or call list(it), either way), that's equivalent to a list comp; if you don't, it's equivalent to a generator expression. Either way, Python doesn't construct each intermediate list before moving on to the next one.
📌On list comprehension vs. method chaining I don't think the aim of this API should be to replace list comprehension and the like. Rather it offers programmers an alternative pattern to transform and reason about their data.
It's not too hard to create your own dataflow class if you want one. It can start with any arbitrary iterable, and then have your map and filter methods just the same. Cool trick: you can even call your class iter! :) class iter: _get_iterator = iter # snapshot the original def __init__(self, basis): self.basis = self._get_iterator(basis) def map(self, func): return type(self)(map(func, self.basis)) # etc def __iter__(self): return self def __next__(self): return next(self.basis) You should then be able to method-chain onto your iter constructors. Personally, I wouldn't use this sort of thing (comprehensions are far easier to work with IMO), but if you want it, the power is in your hands. ChrisA