On Mon, Dec 9, 2019 at 12:48 AM Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
On Sat, 7 Dec 2019 at 00:43, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Dec 06, 2019 at 09:11:44AM -0400, Juancarlo Añez wrote:
[...]
Sure, but in this case, it isn't a fragment of a larger function, and that's not what it looks like. If it looked like what you wrote, I would understand it. But it doesn't, so I didn't really understand what it was supposed to do, until I read the equivalent version using first/next.
Exactly my point.
Indeed, and I agree with that. But I still don't see what advantage there is to having a `first` builtin which does so little. It's a really thin wrapper around `next` that:
calls iter() on its iterable argument supplies a default and then calls next() with two arguments
I guess my question is asking you to justify adding a builtin rather than just educating people how to use next effectively.
The real problem with next is the fact that it raises StopIteration with no default. That can be useful when you are *implementing* iterators but it is very much not what you want when you are just *using* iterators. That makes next something of a footgun because it's tempting to write something like
first = next(iter(iterable))
but if there is no applicable default value that should really be
try: first = next(iter(iterable)) except StopIteration: raise ValueError
If you're defining a first() function, then this would absolutely be correct. I don't think it's a fundamental problem with next(), since its job is to grab the next value from an iterator, or tell you to stop iterating. (BTW, when you're converting exceptions like this in production code, use "raise ValueError from None" to hide the StopIteration from the traceback.)
There is a PEP that attempted to solve this problem: PEP 479 -- Change StopIteration handling inside generators https://www.python.org/dev/peps/pep-0479/
However PEP 479 (wrongly IMO) attributed the problem to generators rather than iterators. Consequently the fix does nothing for users of itertools type functions like map etc. The root of the problem it attempted to fix is the fact that bare next raises StopIteration and so is not directly suitable in situations where you just want to get the next/first element of an iterable.
Hmm. Actually, I'd say that PEP 479 was correct, but that map() is wrong. If you define map() in the most obvious pure-Python way, it will be a generator: def map(func, *iter): while True: args = [next(it) for it in iter] yield func(*args) (modulo some error handling) Written thus, it would be guarded by PEP 479's conversion of StopIteration. I'd say that a more correct implementation of map would be something like: def map(func, *iter): while True: args = [next(it) for it in iter] try: yield func(*args) except StopIteration: raise RuntimeError("mapped function raised StopIteration")
The reason this is particularly pernicious is that it leads to silent action-at-a-distance failure and can be hard to debug. This was considered enough of a problem for PEP 479 to attempt to solve in the case of generators (but not iterators in general).
Agreed, but the problem isn't iterators or next. The problem is with functions that convert iterators into other iterators, while doing other work along the way; if the *other work* raises StopIteration, it causes problems.
This is how I would implement the function in Python:
def first(iterable, default=None): return next(iter(iterable), default)
I agree that that doesn't need to be a builtin. However I would advocate for a function like this:
def first(iterable, *default): iterator = iter(iterable) if default: (default,) = default return next(iterator, default) else: try: return next(iterator) except StopIteration: raise ValueError('Empty iterable')
Ahh the good ol' bikeshedding. The simpler form guarantees that next() is always given a default, ergo it shouldn't ever leak. If you'd prefer it to raise ValueError, then I reckon don't bother implementing the version that takes a default - just let next() do that job, and implement first() the easy way: def first(iterable): it = iter(iterable) try: return next(it) except StopIteration: raise ValueError("Empty iterable")
# silently aborts if any of csvfiles is empty for header in map(lambda e: next(iter(e)), csvfiles): print(header)
(With files, there's no point calling iter, as it'll return the same thing. So you could write this as map(next, csvfiles).)
This kind of confusion can come with iterators and iterables all the time. I can see that the name "first" is potentially confusing. Another possible name is "take" which might make more sense in the context of partially consumed iterators. Essentially the idea should just be that this is next for users rather than implementers of iterables.
Maybe. If it's imported from itertools, though, there shouldn't be any confusion. Since it's somewhat orthogonal to the discussion of first(), I'm going to spin off a separate thread to look at PEP479ifying some iterator conversion functions. ChrisA