On Tue, 12 Oct 2021 at 12:50, Chris Angelico firstname.lastname@example.org wrote:
On Tue, Oct 12, 2021 at 10:24 PM Oscar Benjamin email@example.com wrote:
On Tue, 12 Oct 2021 at 11:48, Chris Angelico firstname.lastname@example.org wrote:
ValueError is no safer. The first() function would have, as its API, "returns the first element or raises ValueError if there is none". So now the caller of first() has to use try/except to handle the case where there is no value. Failing to do so is *just as buggy* as leaking a StopIteration.
A leaky StopIteration is a majorly confusing bug inside a __next__ function, because StopIteration is part of that function's API.
On the contrary: a __next__ function is the only place where it could possibly be valid to raise StopIteration. The fact that next raises StopIteration which passes through to the caller can be useful in this situation and this situation alone: https://github.com/python/cpython/blob/b37dc9b3bc9575adc039c6093c643b7ae5e91...
In any other situation it would be better to call first() and have something like ValueError instead.
Yes, but that's an example of __next__ specifically chaining to next()
- exactly like defining __getattr__ to look for an attribute of
something else (maybe you're writing a proxy of some sort). You expect that a bubbling-up exception is fundamentally equivalent to one you raise yourself.
Please give a real example of where calling first() and getting ValueError is safer than calling next(iter(x)) and getting StopIteration. So far, I am undeterred in believing that the two exceptions have equivalent effect if the caller isn't expecting them.
I think that the situation where I first came across this was something analogous to wanting to separate the header line of a CSV file:
csvfiles = [ ['name', 'joe', 'dave'], ['name', 'steve', 'chris'], , # whoops, empty csv file ['name', 'oscar'], ]
def remove_header(csvfile): it = iter(csvfile) next(it) return it
# print all names from all csv files for names in map(remove_header, csvfiles): for name in names: print(name)
If you run the above you get
$ python t.py joe dave steve chris
The data following the empty file (i.e. "oscar") was silently discarded. The context where I found this was something that took much longer to run and was harder to check and debug etc. I have not personally made the same mistake again because I have since been automatically wary of any usage of next(). I couldn't possibly count the number of times I've seen unsafe usage of next in code suggestions on mailing lists like this though (see all the examples above in this thread).
The problem is that the erroneous case which is the empty file leads to a StopIteration. Unlike a normal exception though, a StopIteration can be caught without try/except. In the above it is the *for-loop* that swallows the exception. Had it been literally any other exception type then you would have been looking at a Traceback instead of silently discarded data:
$ python t.py joe dave steve chris Traceback (most recent call last): File "t.py", line 21, in <module> for names in map(remove_header, csvfiles): File "t.py", line 18, in remove_header first(it) File "t.py", line 14, in first raise ValueError from None ValueError
Your suggestion is that this is a bug in map() which is a fair alternative view. Following through to its conclusion your suggestion is that every possible function like map, filter, and all the iterator implementations in itertools and in the wild should carefully wrap any internal non-next function call in try/except to change any potential StopIteration into a different exception type.
My view is that it would be better to have a basic primitive for getting an element from an iterable or for advancing an iterator that does not raise StopIteration in the first place. I would probably call that function something like "take" rather than "first" though. The reason I prefer introducing an alternative to next() is because I think that if both primitives were available then in the majority of situations next() would not be the preferred option.