Can we add "zip and assert equal length" to the standard library?

I find that about 90% of the time I want want to zip iterators together, I expect them to be the same length and want to throw an exception if they aren't. Yet there is not currently a solution for this in the standard library for this, and as a result I always have to take this function everywhere I go: def zip_equal(*iterables): """ Zip and raise exception if lengths are not equal. Taken from solution by Martijn Pieters, here: http://stackoverflow.com/questions/32954486/zip-iterators-asserting-for-equa... :param iterables: Iterable objects :return: A new iterator outputting tuples where one element comes from each iterable """ sentinel = object() for combo in zip_longest(*iterables, fillvalue=sentinel): if any(sentinel is c for c in combo): raise ValueError('Iterables have different lengths. Iterable(s) #{} (of 0..{}) ran out first.'.format([i for i, c in enumerate(combo) if c is sentinel], len(combo)-1)) yield combo Would anybody object to adding this to the standard library for Python 3.8?

Hi Peter Thank you for your deferred default values idea, which we're now working on together. https://github.com/petered/peters_example_code/blob/master/peters_example_co... You wrote:
I find that about 90% of the time I want want to zip iterators together, I expect them to be the same length and want to throw an exception if they aren't. Yet there is not currently a solution for this in the standard library for this, and as a result I always have to take this function everywhere I go [function definition snipped]
You need this function wherever you go. You can solve that problem right now by putting it into PyPI. And if you put it on github, perhaps someone will fork it and contribute. (Note to self. I should do the same for my own useful little tools.) You asked about putting in the standard library. There's a process for that. https://www.python.org/dev/peps/pep-0001/ You might also want to look at the PEP index, to find a successful PEP that's similar to yours. https://www.python.org/dev/peps/ Here's one I've found that looks similar. It proposes the inclusion of a third-party module, pathlib, in the standard library. https://www.python.org/dev/peps/pep-0428/ Putting your code into PyPI into PyPI solves your immediate problem. It also also gives you a reference implementation. https://www.python.org/dev/peps/pep-0001/#what-belongs-in-a-successful-pep I hope your next post will be URL for code on github. I'd be interested in making a fork. Best wishes Jonathan

On Fri, Jul 27, 2018 at 07:02:37PM +0200, Peter O'Connor wrote:
I find that about 90% of the time I want want to zip iterators together, I expect them to be the same length and want to throw an exception if they aren't. Yet there is not currently a solution for this in the standard library for this, and as a result I always have to take this function everywhere I go: [snip]
Indeed. The need is real, and the question has come up many times on Python-List as well.
Would anybody object to adding this to the standard library for Python 3.8?
Given that the obvious place to put it is itertools, the person you need to convince is Raymond Hettinger. -- Steve

I have never wanted this behavior myself. So that's 0% of the time for me. I happily believe that Peter O'Connor wants it 90% of the time... although I guess it suggests he probably organizes his algorithms differently than I do at some broader level. For my needs, zip() is common, and itertools.zip_longest() is somewhat uncommon, but not unheard of. I do note that https://more-itertools.readthedocs.io also does not have this zip_equal() functionality. At least I don't see it (I could have missed it under a different name, but I looked around). This is pretty clearly the most widely used "extra stuff that might go in itertools" library. I think part of the problem is that raising an exception when something is exhausted (other than StopIteration) is contrary to the spirit of itertools (or more_itertools). I think part of the reason those iterator functions do not do that is that an exception often indicates "something went wrong," but if this problem occurs after an indefinitely long iteration that is later than you'd like to know. A more common pattern, in my experience would be to put an exception in the higher-up block where an iterator is consumed. Various "things went wrong" conditions can be checked, not exclusively that one iterator ran out before the other. On Fri, Jul 27, 2018 at 1:03 PM Peter O'Connor <peter.ed.oconnor@gmail.com> wrote:
I find that about 90% of the time I want want to zip iterators together, I expect them to be the same length and want to throw an exception if they aren't. Yet there is not currently a solution for this in the standard library for this, and as a result I always have to take this function everywhere I go:
def zip_equal(*iterables): """ Zip and raise exception if lengths are not equal.
Taken from solution by Martijn Pieters, here:
http://stackoverflow.com/questions/32954486/zip-iterators-asserting-for-equa...
:param iterables: Iterable objects :return: A new iterator outputting tuples where one element comes from each iterable """ sentinel = object() for combo in zip_longest(*iterables, fillvalue=sentinel): if any(sentinel is c for c in combo): raise ValueError('Iterables have different lengths. Iterable(s) #{} (of 0..{}) ran out first.'.format([i for i, c in enumerate(combo) if c is sentinel], len(combo)-1)) yield combo
Would anybody object to adding this to the standard library for Python 3.8?
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

This is a functionality I sometimes need. Maybe you can do a pull request to more-itertools and that would be the end of it? I don't know if that's general enough for being added to the standard library, more-itertools seems the way to go for me. Go find out if raising a ValueError suits their Api (at least one of their function, "first" does it). The David's comment makes senses, what happens when you pass an infinite iterable, will it be consumed half? That seems logical. from itertools import count it = count(0) try: L = zip_equal(range(5), it) except ValueError: print(next(it)) # prints 6 but I'd expect 5 However, maybe that's irrelevant because most of the time you don't care about the iterable when the exception is raised. Le sam. 28 juil. 2018 à 01:54, David Mertz <mertz@gnosis.cx> a écrit :
I have never wanted this behavior myself. So that's 0% of the time for me. I happily believe that Peter O'Connor wants it 90% of the time... although I guess it suggests he probably organizes his algorithms differently than I do at some broader level. For my needs, zip() is common, and itertools.zip_longest() is somewhat uncommon, but not unheard of.
I do note that https://more-itertools.readthedocs.io also does not have this zip_equal() functionality. At least I don't see it (I could have missed it under a different name, but I looked around). This is pretty clearly the most widely used "extra stuff that might go in itertools" library.
I think part of the problem is that raising an exception when something is exhausted (other than StopIteration) is contrary to the spirit of itertools (or more_itertools). I think part of the reason those iterator functions do not do that is that an exception often indicates "something went wrong," but if this problem occurs after an indefinitely long iteration that is later than you'd like to know.
A more common pattern, in my experience would be to put an exception in the higher-up block where an iterator is consumed. Various "things went wrong" conditions can be checked, not exclusively that one iterator ran out before the other.
On Fri, Jul 27, 2018 at 1:03 PM Peter O'Connor <peter.ed.oconnor@gmail.com> wrote:
I find that about 90% of the time I want want to zip iterators together, I expect them to be the same length and want to throw an exception if they aren't. Yet there is not currently a solution for this in the standard library for this, and as a result I always have to take this function everywhere I go:
def zip_equal(*iterables): """ Zip and raise exception if lengths are not equal.
Taken from solution by Martijn Pieters, here:
http://stackoverflow.com/questions/32954486/zip-iterators-asserting-for-equa...
:param iterables: Iterable objects :return: A new iterator outputting tuples where one element comes from each iterable """ sentinel = object() for combo in zip_longest(*iterables, fillvalue=sentinel): if any(sentinel is c for c in combo): raise ValueError('Iterables have different lengths. Iterable(s) #{} (of 0..{}) ran out first.'.format([i for i, c in enumerate(combo) if c is sentinel], len(combo)-1)) yield combo
Would anybody object to adding this to the standard library for Python 3.8?
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi All I've followed my own advice, earlier in this thread. The newly created repository https://github.com/jfine2358/py-jfine2358 contains personal Python code that others might find useful. And you'll find in it: === https://github.com/jfine2358/py-jfine2358/blob/master/jfine2358/itertools.py def zipclose(*argv, close): '''As zip(*argv), but call close before raising StopIteration. [snip]''' === Given zipclose, we can solve Peter's problem by writing a suitable close function. And some other problems can be solved, by using a different close function. Comment on my code at https://github.com/jfine2358/py-jfine2358/issues (preferred), or on this thread. Finally, thank you Peter for another good problem. -- Jonathan
participants (5)
-
David Mertz
-
Jonathan Fine
-
Peter O'Connor
-
Robert Vanden Eynde
-
Steven D'Aprano