On Sat, 7 Dec 2019 at 00:43, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Dec 06, 2019 at 09:11:44AM -0400, Juancarlo Añez wrote:
[...]
Sure, but in this case, it isn't a fragment of a larger function, and that's not what it looks like. If it looked like what you wrote, I would understand it. But it doesn't, so I didn't really understand what it was supposed to do, until I read the equivalent version using first/next.
Exactly my point.
Indeed, and I agree with that. But I still don't see what advantage there is to having a `first` builtin which does so little. It's a really thin wrapper around `next` that:
calls iter() on its iterable argument supplies a default and then calls next() with two arguments
I guess my question is asking you to justify adding a builtin rather than just educating people how to use next effectively.
The real problem with next is the fact that it raises StopIteration with no default. That can be useful when you are *implementing* iterators but it is very much not what you want when you are just *using* iterators. That makes next something of a footgun because it's tempting to write something like first = next(iter(iterable)) but if there is no applicable default value that should really be try: first = next(iter(iterable)) except StopIteration: raise ValueError There is a PEP that attempted to solve this problem: PEP 479 -- Change StopIteration handling inside generators https://www.python.org/dev/peps/pep-0479/ However PEP 479 (wrongly IMO) attributed the problem to generators rather than iterators. Consequently the fix does nothing for users of itertools type functions like map etc. The root of the problem it attempted to fix is the fact that bare next raises StopIteration and so is not directly suitable in situations where you just want to get the next/first element of an iterable. So you can have something like this: csvfiles = [ ['header', '1', '2', '3'], [], # <----- file has no header ['header', '4', '5', '6'], # This file is skipped ] def total_csvfile(lines): lines = iter(lines) header = next(lines) # Skip header return sum(int(row) for row in lines) for total in map(total_csvfile, csvfiles): print(total) This prints out the total of the first csv file. Then StopIteration that is emitted from attempting to skip the missing header of the second csvfile. That StopIteration leaks out from map.__iter__ and is "caught" by the enclosing for loop. If you change the end of the script to totals = map(total_csvfile, csvfiles) for total in totals: print(total) for total in totals: print(total) then you will see totals for the files after the empty file showing that it is the for loop that caught the StopIteration. The reason this is particularly pernicious is that it leads to silent action-at-a-distance failure and can be hard to debug. This was considered enough of a problem for PEP 479 to attempt to solve in the case of generators (but not iterators in general).
This is how I would implement the function in Python:
def first(iterable, default=None): return next(iter(iterable), default)
I agree that that doesn't need to be a builtin. However I would advocate for a function like this: def first(iterable, *default): iterator = iter(iterable) if default: (default,) = default return next(iterator, default) else: try: return next(iterator) except StopIteration: raise ValueError('Empty iterable') This has the following behaviour:
first({1, 2, 3}) 1 first(x for x in [1, 2]) 1 first([]) Traceback (most recent call last): ... ValueError: Empty iterable first([], 2) 2
You can use it safely with map e.g. to get the first element of a bunch of iterables: # raises ValueError if any of csvfiles is empty for header in map(first, csvfiles): print(header) With next that would be # silently aborts if any of csvfiles is empty for header in map(lambda e: next(iter(e)), csvfiles): print(header)
But there's a major difference in behaviour depending on your input, and one which is surely going to lead to bugs from people who didn't realise that iterator arguments and iterable arguments will behave differently:
# non-iterator iterable py> obj = [1, 2, 3, 4] py> [first(obj) for __ in range(5)] [1, 1, 1, 1, 1]
# iterator py> obj = iter([1, 2, 3, 4]) py> [first(obj) for __ in range(5)] [1, 2, 3, 4, None]
We could document the difference in behaviour, but it will still bite people and surprise them.
This kind of confusion can come with iterators and iterables all the time. I can see that the name "first" is potentially confusing. Another possible name is "take" which might make more sense in the context of partially consumed iterators. Essentially the idea should just be that this is next for users rather than implementers of iterables. -- Oscar