On Sun, 17 May 2020 at 07:10, Christopher Barker wrote:

...

PS. Why wasn't a new builtin zip_strict() on the menu? I think I would have given it at least +0.5, because of this rule of thumb.

I would think that if zip_strict() added as a builtin, then zip_longest() should too.

And the fact that zip_longest was not added as a builtin made me think that it was a non-starter.

I think that's just because zip_longest isn't a very compelling alternative to zip. I've known of it for a long time and I don't remember *ever* using it. If builtins had zip_shortest (i.e. current zip), zip_strict and zip_longest then I think I would use zip_strict 95% of the time, zip_shortest 5% of the time and zip_longest 0% of the time. -- Oscar

On Sun, May 17, 2020 at 6:14 AM Rhodri James wrote:

On 16/05/2020 17:14, Guido van Rossum wrote:

...

On Sat, May 16, 2020 at 1:26 AM Steven D'Aprano wrote:

...

...

* zip(strict=True) +1 * zip(mode='strict') -0 * itertools.zip_strict() -0.5 * zip.strict() -1 (but really, I'd like to make this -1e10) I spent a significant amount of time and mental energy explaining in detail why a boolean flag is a poor API and is objectively the wrong interface here. This is not just a matter of personal taste: there are reasons why a flag is wrong here.

Clearly it is not the objective truth, otherwise you would have an easier way convincing everyone.:-)

OK, let's put some numbers on this. We only have 9 votes in, but aside from Brandt (whose position is fairly obvious from the PEP) that includes most of the people who have expressed strong opinions one way or another. Ignoring the nuances of +/-0, we end up with:

itertools.zip_strict(): +5.5 zip(strict=True): +1 zip.strict(): -1.9 zip(mode='strict'): -4

(I bet Steven is wishing he hadn't been so generous to the "strict=True" option now :-)

Now I'm not fool enough to mistake a public vote for an objective truth (::looks pointedly around world politics::) but there are some interesting conclusions here.

1) Nobody (aside from Steven) wants a "mode" keyword. I suspect this is because both of the two main camps (zip(strict=True) and zip_strict()) have solid reasons against it; for the first, it's an unnecessary distraction, and for the second the existence of zip_longest() argues against it.

2) People don't hate zip.strict() as much as I had expected.

3) The PEP needs to come up with an actual argument against itertools.zip_strict(). The current dismissal ain't going to cut it.

Let me attempt a metaphor, which won't be perfect but may help: The safety one gets from strictness is a bit like driving a car wearing a seat belt. It is not fundamentally different from driving a car without a seat belt, and in most cases you hope the seat belt will not come into play. But it is a precaution that is worth taking in *most* circumstances (not all, e.g. for infants a standard seat belt won't work). The built-in approaches (whether spelled as zip(strict=True), or zip.strict(), or a new zip_strict() builtin) make it really easy for everybody to take the safety precaution where it is appropriate. By contrast, putting it in itertools where it has to be imported is like requiring someone to rent seat belts in order to use them in their car. Some safety-conscious folks will go to the trouble; others won't. itertools.zip_longest() is sort of a power tool for specialized situations—in our analogy, say, a pickup truck that has to be rented. Yes, with some work it *can* be used to emulate strict-zip, but most people won't think of it that way; they will think of it as something you only need in special situations. Is there some logic to the objection that it is weird to have two forms of zip (or one form with two variants) that are built in, and a third that is in itertools? Sure. But this seems to me a clear case of practicality beats purity. As an extremely common use case of zip (for many people), it will be most useful if it is built in. Nathan

-- Rhodri James *-* Kynesim Ltd _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/N4QLAL... Code of Conduct: http://python.org/psf/codeofconduct/

On 05/17/2020 10:18 AM, Alex Hall wrote:

But it's good that we have the assert statement, because it makes it easy to write safe code and so people are encouraged to do so.

I could not disagree more strongly with this. Every time I have seen assert used it was in such a way that the program could easily silently produce erroneous results or fail far from the error if asserts were turned off. -- ~Ethan~

On Sun, May 17, 2020 at 12:22 PM Nathan Schneider wrote:

Let me attempt a metaphor, which won't be perfect but may help:

The safety one gets from strictness is a bit like driving a car wearing a seat belt. It is not fundamentally different from driving a car without a seat belt, and in most cases you hope the seat belt will not come into play. But it is a precaution that is worth taking in *most* circumstances (not all, e.g. for infants a standard seat belt won't work).

That's a really terrible analogy. :-( I never drive without a seat belt. And a never want the seat belt to actually matter, of course. Everyone who want a zip_strict behavior (including me) wants to be able either to catch the exception explicitly or to have the program fail-fast/fail-hard because of it. In contrast, as I've said, more than half of the times that *I* use zip() it would be BROKEN by using zip_strict() instead (or zip(..., strict=True), or whichever spelling). Raising an exception for something I want to succeed, and I want to work exactly as it does (e.g. leave some iterators unconsumed) is not a "harmless safety precaution". If you want a better metaphor: Some door handles include locks, others do not. "Strict" ones have locks. So yes, it's possible to leave the lock in the unlocked position, and then it functions pretty much the same as one without a lock. But likewise, it's possible to leave the door in the locked position when you don't have the key on you, and you face a significant inconvenience that serves no purpose. I have some doors with locks, and some other doors without locks. I have both for a good reason, but the reasons are different, and depend on various things like whether a particular room is private or contains valuables. In truth though, I don't lock my outside doors because I live in a community of "consenting adults" (occasionally I do lock the bathroom door for privacy, for a short while... no-locks is definitely strongly my default mode, as is no-strict when I zip).

--

The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.

On Sun, May 17, 2020 at 2:09 PM Nathan Schneider wrote:

...

not. "Strict" ones have locks. So yes, it's possible to leave the lock in the unlocked position, and then it functions pretty much the same as one without a lock. But likewise, it's possible to leave the door in the locked position when you don't have the key on you, and you face a significant inconvenience that serves no purpose.

For some of us, strictness is a property that users often want when they use zip(), whether they are properly enforcing it or not—so giving them a really easy way to opt into it would help avoid bugs. (Personally, I cannot remember the last time I actually wanted non-strict zip.) I think David is saying that he more often wants non-strict zip, and it would be tempting and dangerous to make strict-zip too easy to use for

If you want a better metaphor: Some door handles include locks, others do those who aren't thinking carefully about their code, so it would be better to bury strict-zip in itertools for the power users who really know they need it. (Is that a fair summary?)

The API matter is really orthogonal to this. My point here is that Nathan and some other discussants are operating under the assumption that: "Everyone really wants strict-zip but they just haven't had a way to express it conveniently. They all assume their iterables are the same length already, so this just adds a check." I disagree strongly with that assumption. I think that the actual majority of my uses of zip are non-strict. Admittedly, I have not scanned my code to count that; for that matter, most of the code I have written is no longer accessible to me, being written for companies I no longer work for (and not open source). But whatever the actual percentages might be, I COMMONLY want a non-strict zip by actual specific intention, not because I've made a wrong assumption about the nature of the iteratables I use. Of course, I also commonly use zip with the implicit assumption that my iterables are the same length... I have most certainly written many lines where I would appropriately choose strict-zip if it existed (whichever API). To me, itertools is not some hidden vault only accessible after performing Herculean labors. I believe boolean mode switches are usually a bad design for Python. Not always, there are exceptions like open(). And I think Guido made the good point that one of the things that makes mode switches bad is when they change the return type, which the `strict=True` API would not do (but it would create a new kind of exception to worry about). In fact, itertools is pretty much the only module where I occasionally write `from itertools import *`. There are many good things in that module that are general purpose. But namespaces, after all, are a honkin' good idea. I think if `zip()` were proposed today as a brand new function that hadn't existed, I would advocate strongly for putting it inside itertools. Probably `map()` and `filter()` similarly. So I don't want zip_strict() to join built-ins, but it's not because I think it is a niche case, but rather because I think we already have more in built-ins than we should, and some of it could very reasonably live in a more descriptive namespace. I would certainly not mind if we added `zip_shortest()` as a synonym for the current zip to itertools. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.

On 17/05/2020 19:43, David Mertz wrote:

On Sun, May 17, 2020 at 2:09 PM Nathan Schneider mailto:neatnate@gmail.com> wrote:

If you want a better metaphor: Some door handles include locks, others do not.  "Strict" ones have locks.  So yes, it's possible to leave the lock in the unlocked position, and then it functions pretty much the same as one without a lock.  But likewise, it's possible to leave the door in the locked position when you don't have the key on you, and you face a significant inconvenience that serves no purpose.

For some of us, strictness is a property that users often want when they use zip(), whether they are properly enforcing it or not—so giving them a really easy way to opt into it would help avoid bugs. (Personally, I cannot remember the last time I actually wanted non-strict zip.) I think David is saying that he more often wants non-strict zip, and it would be tempting and dangerous to make strict-zip too easy to use for those who aren't thinking carefully about their code, so it would be better to bury strict-zip in itertools for the power users who really know they need it. (Is that a fair summary?)

The API matter is really orthogonal to this.  My point here is that Nathan and some other discussants are operating under the assumption that: "Everyone really wants strict-zip but they just haven't had a way to express it conveniently. They all assume their iterables are the same length already, so this just adds a check."

I disagree strongly with that assumption.  I think that the actual majority of my uses of zip are non-strict. David, you have made this point at great length - we get it. But it seems clear to me (YMMV) that you are in a small minority. And nobody is suggesting changing the default behaviour of zip. Your code won't break. But a lot of users (including you, in a minority of cases) can, *if they choose*, benefit from a 'strict' version. So what is wrong (ceteribus paribus) with making it easy for them to do so?

On Sun, May 17, 2020 at 02:43:52PM -0400, David Mertz wrote:

To me, itertools is not some hidden vault only accessible after performing Herculean labors.

+1 to David's observation here. I find it remarkable how people on this list who often argue "just put it on PyPI" as if that didn't condemn the proposal to die are now arguing that importing from itertools is an undue burden.

I believe boolean mode switches are usually a bad design for Python.

I don't think that Python is unique in that regard.

Not always, there are exceptions like open().

As Rhodi points out in another message, `open` is not an exception. In fact the opposite: `open` is an excellent example of *not* using bool flags. We have this: open(file, mode='rt', ...) not this: open(file, read=True, write=False, exclusive=False, append=False, binary=False, text=True, update=False, universal=True, ...) There are cases where bool flags are acceptable, even preferred. For example, the reverse=False parameter to sorted() seems to be okay because it is independent of, and orthogonal to, any other sorting parameters such as the key function, or the cmp function in Python 2. If you can compose the behaviour of the flag with the other parameters, it might be okay. For example, back to open: open(..., closefd=True, ...) seems to be fine, since it is independent of any of the value of the other parameters. You can pass closefd=False, or True, regardless of everything else. (There is a technical limitation that it must be True if the file is given by name, but it is independent of everything else.) This proposed mode is not a composable flag like reverse or closefd. If we treat it as a flag instead of a mode, then we either rule out future enhancements to zip (they *must* go into new functions), or commit to piling bool flag upon bool flag even though most of the combinations will be an error. E.g. there were proposals to make `shortest` an explicit parameter. You can't have `zip(shortest=True, strict=True, longest=True, ...)`.

And I think Guido made the good point that one of the things that makes mode switches bad is when they change the return type, which the `strict=True` API would not do (but it would create a new kind of exception to worry about).

I think that the return type is a red herring here. Here's a strawman function to demonstrate that the return type is not very important in and of itself: def encode(obj, text=True): if text: return json.dumps(obj) else: return pickle.dumps(obj).decode('latin1') Note: json.dumps is another case where the bool flags are acceptable, because they all control orthogonal, independent features of the call. Passing skipkeys=True doesn't prevent you from also passing ensure_ascii=False. In contrast, you can't compose strict=True with other zip modes. -- Steven

On Wed, May 20, 2020 at 06:31:51PM -0700, Christopher Barker wrote:

I'm still confused why the ternary flag (mode) idea never comes up in these arguments -- I know I like that the best. But yeah, I can except that it's dead.

Because the standard spelling of flags in ternary logic are True, False, and either Unknown or Maybe, and both `zip(*args, strict=Unknown)` and `strict=Maybe` are awful. Because Python doesn't have builtin ternary flags. Because the usual work-around for that, forcing the falsey value None to mean Unknown or Maybe, is rather odd; but even if it wasn't, `strict=None` is hardly a self-explanatory API. Because a three-state flag is not extensible past three behaviours. (And I'm not even sure I know what you want as the third behaviour.) If you want a *mode* parameter, the modes can be mnemonics for the behaviour: mode='strict' # not tolerant of length mismatches mode='shortest' # stops at the shortest argument; # possible future enhancements mode='longest' # pads to the longest argument mode='skip' # skips arguments as they become empty mode='that thing with StopIteration that Soni wants' The modes wouldn't have to be strings, they could be enums, although we might not want them to be builtins. -- Steven

On 16/05/2020 17:14, Guido van Rossum wrote:

On Sat, May 16, 2020 at 1:26 AM Steven D'Aprano mailto:steve@pearwood.info> wrote:

> * zip(strict=True)       +1 > * zip(mode='strict')     -0 > * itertools.zip_strict() -0.5 > * zip.strict()           -1  (but really, I'd like to make this -1e10)

I spent a significant amount of time and mental energy explaining in detail why a boolean flag is a poor API and is objectively the wrong interface here. This is not just a matter of personal taste: there are reasons why a flag is wrong here.

[snip]

- your preferred option is the least open to future enhancements;

Given zip's stability, I doubt there will be a lot of other future enhancements any time soon. In Python's culture, boolean flag is the most common way to modify the behavior of a function. The reasons have to do with tradition (lots of existing APIs use this pattern) as well as ease of implementation (*also* in the Zen!), and also with how people *think* about certain APIs. Zip-strict is "like zip, but strict(er) in its requirements".

But Steven has a point.  zip(mode='strict') would allow zip_longest behaviour to be incorporated into zip at some future time, should that be considered desirable. (And conceivably other behaviours that might crop up.  I don't have any use cases, but one possibility might be "stop zipping after N items".) Whereas doing that with a boolean `strict` would lead, as others have pointed out, to an ugly API (2 booleans that can't both be True). Rob Cliffe