On 2020-04-21 11:20 a.m., Eric V. Smith wrote:

...

On 4/21/2020 10:10 AM, Soni L. wrote:

...

I feel like zip could return partial results:

try:   next(zip([0], [])) except StopIteration as exc:   assert StopIteration.args == (0,)

how much would this break?

It would break a lot of code, and so it won't happen. Actually, I'm not so sure of that, presuming you mean "exc.args", not "StopIteration.args". And since that's currently set to "()", at least in my tests, maybe setting it wouldn't break much code. But it's

On 4/21/2020 10:14 AM, Eric V. Smith wrote: probably a non-zero amount.

... yeah I did not pay as much attention to that hypothetical code as I should have.

...

You can use itertools.zip_longest with your own sentinel to get the same behavior.

In your proposal, what would next(zip([0], [], [0])) result in?

This point still stands.

it'd be the partial results. by the time zip sees the StopIteration it has only seen the first iterator, so you get (0,) and the third iterator doesn't advance. if you had stored the third iterator somewhere, you'd be able to call next on it directly as well.

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BBDE2X... Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Apr 22, 2020 at 1:22 AM Soni L. <fakedme+py@gmail.com> wrote:

looking further into it, zip() already guarantees the StopIteration isn't gonna preserve the original StopIteration, so this should be a non-breaking change:

https://docs.python.org/3/library/functions.html#zip

I'm sorry, I'm not sure I understand you here - do you mean that it *doesn't* guarantee that it *will* preserve it, or is there actually a guarantee that it will *not* preserve it?

Equivalent to:

Bear in mind that the equivalencies aren't always exact.

result = [] for it in iterators: elem = next(it, sentinel) if elem is sentinel: return result.append(elem)

my proposal is to change that "return" into "return tuple(result)".

That'd make it show up as the exception's 'value', not in its args. If indeed there's no guarantee, then I would fully support this change - should be a non-breaking change and could be convenient. ChrisA

On Wed, Apr 22, 2020 at 8:20 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Apr 22, 2020 at 01:25:14AM +1000, Chris Angelico wrote:

...

That'd make it show up as the exception's 'value', not in its args. If indeed there's no guarantee, then I would fully support this change - should be a non-breaking change and could be convenient.

Convenient for what?

I'm assuming that the OP has a use-case. I'm just saying that this way wouldn't need to break any code, so it would be a plausible change. ChrisA

On Wed, Apr 22, 2020 at 12:25 AM Eric V. Smith <eric@trueblade.com> wrote:

...

On 4/21/2020 10:10 AM, Soni L. wrote:

...

I feel like zip could return partial results:

try: next(zip([0], [])) except StopIteration as exc: assert StopIteration.args == (0,)

how much would this break?

It would break a lot of code, and so it won't happen. Actually, I'm not so sure of that, presuming you mean "exc.args", not "StopIteration.args". And since that's currently set to "()", at least in my tests, maybe setting it wouldn't break much code. But it's

On 4/21/2020 10:14 AM, Eric V. Smith wrote: probably a non-zero amount.

Very minor bikeshedding: Since args is normally going to have a predictable length, I think it'd be better to declare that the first argument is a truncated tuple. That means wrapping it up in one more level of tuple, but in return, the raised exception has the option to have other arguments added in the future. Alternatively, StopIteration already has a 'value' attribute (used for the return value of a generator). Would it be better to use that rather than the args? Currently, it retains any value that was already in it - so if the shortest iterable is a generator, then its return value will be in the propagated StopIteration. Not sure if that's a guarantee and therefore covered by backward compatibility. ChrisA

22.04.20 08:11, Greg Ewing пише:

I don't this should be a guarantee, in this or any similar situation. The only purpose for the value in a StopIteration is to provide a return value for yield-from when the thing being delegated to is a generator. If you're yielding from something that's not a generator, and not specifically designed to mimic one, then all bets are off.

Several years ago there was a discussion about extending itertools to support generators (passing throw the return value, supporting send() and throw(), etc). This idea was rejected at that time due to a lack of good use cases.

On Tue, Apr 21, 2020 at 05:33:24PM -0300, Soni L. wrote:

1. see the other thread (strict zip), especially the parts where they brought up the lack of peekable/unput iterators in the context of getting a count out of an iterator.

I've seen it, as far as I can tell there are already good solutions for getting a count out of an iterator.

2. it'd help me clean up some of my code. :p

I don't see how it would clean up code rather than make it uglier. Can you give an example?

3. I feel like it makes intuitive sense for zip to return a partial result somehow, especially now that StopIteration and return work together.

With respect Soni, your intuitions do not have a good track record of either being shared by other people, or of making good programming features. Maybe it's time for you to stop suggesting things that make "intuitive sense" and start putting some careful, logical thought into the proposal instead of going by intuition? -- Steven

On 22.04.20 06:43, Soni L. wrote:

On 2020-04-21 7:12 p.m., Steven D'Aprano wrote:

...

On Tue, Apr 21, 2020 at 05:33:24PM -0300, Soni L. wrote:

...

1. see the other thread (strict zip), especially the parts where they > brought up the lack of peekable/unput iterators in the context of > getting a count out of an iterator.

I've seen it, as far as I can tell there are already good solutions for getting a count out of an iterator.

are there *any* solutions for getting partial results out of zip with different-length iterators of unknown origin?

You can basically use the code from this StackOverflow answer (code attached below) to cache the last object yielded by each iterator: https://stackoverflow.com/a/61126744     iterators = [         iter([0, 1, 2]),         iter([3, 4]),         iter([5]),         iter([6, 7, 8])     ]     iterators = [cache_last(i) for i in iterators]     print(list(zip(*iterators)))   # [(0, 3, 5, 6)]     partial = []     for i in iterators:         try:             partial.append(i.last)         except StopIteration:             break     print(partial)   # [1, 4] This is the code for `cache_last`:     class cache_last:         def __init__(self, iterable):             self.obj = iterable             self._iter = iter(iterable)             self._sentinel = object()         @property         def last(self):             if self.exhausted:                 raise StopIteration             return self._last         @property         def exhausted(self):             if not hasattr(self, '_last'):                 raise ValueError('Not started!')             return self._last is self._sentinel         def __next__(self):             try:                 self._last = next(self._iter)             except StopIteration:                 self._last = self._sentinel                 raise             return self._last         def __iter__(self):             return self

On 22.04.20 11:19, Steven D'Aprano wrote:

On Wed, Apr 22, 2020 at 10:52:44AM +0200, Dominik Vilsmeier wrote:

...

You can basically use the code from this StackOverflow answer (code attached below) to cache the last object yielded by each iterator: https://stackoverflow.com/a/61126744 Caching the result of iterators is unsafe if the value yielded depends on the environment at the time. It can leave you vulnerable to Time Of Check To Time Of Use bugs, or inaccurate results.

Good point, but in this case it only caches the most recently yielded value. It's the value that would have ended up in a tuple during `zip` if not one of the other iterators had stopped the iteration. So whatever state the environment is in, it still corresponds to those cached values. How this partial result is then used is a different question.

On 2020-04-22 6:47 a.m., Steven D'Aprano wrote:

On Wed, Apr 22, 2020 at 01:43:10AM -0300, Soni L. wrote:

...

are there *any* solutions for getting partial results out of zip with different-length iterators of unknown origin?

No. If you want to continue getting results even when one or more of the iterators is exhausted, `zip` is the wrong tool. Use `zip_longest`.

...

...

...

3. I feel like it makes intuitive sense for zip to return a partial result somehow, especially now that StopIteration and return work together. [...]

...

it makes intuitive sense that there'd be some way to get the iterator elements silently swallowed by zip against the wishes of the programmer.

If you use zip, it is because you want zip's behaviour, which is explicitly documented as truncating on the shortest iterator. (Either that or you don't pay attention to the docs.)

You can't say "its against the wishes of the programmer". Nobody is forcing you to use zip except yourself. If you are hammering nails with a screwdriver, that's not the screwdriver's fault.

As I said, if you want to continue collecting values even after one or more iterators are exhausted, use zip_longest.

...

there isn't. so we avoid it because it's useless for anything more advanced than the "you've checked the lengths beforehand" use-case.

I frequently use zip all the time with infinite iterators, or one infinite iterator like itertools.count() and one or more finite iterators, or more than one finite iterator, and I've never checked their lengths ahead of time.

Your statement that zip is "useless" unless you have checked the lengths first is simply nonsense.

I have, more than once, had a desire to use zip to partially consume an iterator from a set of iterators, and then pass the rest onto something else. it'd fit in with that. it's a small quality of life improvement that'd make zip even more useful than it is today, and I'm pretty sure it can't break anything because zip (as is documented) already puts None in the StopIteration. (whether the implementation follows that is another story, I haven't checked, but that'd be even more of a reason to change this.) not sure why so much pushback from you.

On 04/22/2020 07:08 AM, Soni L. wrote:

I have, more than once, had a desire to use zip to partially consume an iterator from a set of iterators, and then pass the rest onto something else. it'd fit in with that.

it's a small quality of life improvement that'd make zip even more useful than it is today, and I'm pretty sure it can't break anything because zip (as is documented) already puts None in the StopIteration. (whether the implementation follows that is another story, I haven't checked, but that'd be even more of a reason to change this.) not sure why so much pushback from you.

Because it's one more thing that would require support, that would only be rarely used. Specialized bits are usually left to third-parties and not put in the stdlib without mitigating circumstances. In this case there are enough possible specialized uses and requirements and desired outcomes that the stdlib should stay out of it. -- ~Ethan~