On 21 July 2015 at 21:19, Paul Moore wrote:

On 21 July 2015 at 11:03, Ben Finney wrote:

...

* If challenged to do so, could one (the contributor) present a compelling justification for the change?

This is what I claim Paul Moore's doubt (fear?) is indicative of. I maintain that this doubt is quite healthy: it helps the contributor to pause, reflect, seek assistance in making decisions, and thereby also tends to exclude poorly-justified changes which would otherwise be committed.

That is *not* what I was trying to express. My fear is that I will be subjected to the sort of unreasonable level of debate and frankly criticism that came up in this thread, and I'm not sure if I have the energy to deal with it. I wouldn't ever commit something unless *in my judgement* it was OK to go in. Whether my judgement is sufficient is the whole point here.

Exactly. Being granted a commit bit means "The folks that got CPython to where it is today trust you to take appropriate risks in guiding CPython's future". It can be safely assumed that *everyone* that has that authority takes it *very* seriously (otherwise they wouldn't have been granted commit privileges). If there weren't any risks to be weighed and judgment calls to be made, we could let a computer decide what commits should go in (and indeed "make patchcheck" already handles some of those basic tasks, and we'd like to automate even more of them like "does this patch apply, and do the tests still pass on x86_64 under at least Windows and a long term support Linux distro after applying it?"). This challenge is reflected in the fact that the Zen of Python is deliberately self-contradictory, as it articulates competing design principles to take into consideration, rather than being able to provide ironclad rules that avoid the need for human judgement in determining *which* of those design guidelines are most salient in any given situation. Now, proposing further *enhancements* to a change is perfectly reasonable, especially if the feedback is based on practical experience with the original version of the change. The implementation of PEP 492 (async/await), for example, underwent some fundamental changes during the beta process, as developers' feedback showed that some of the design decisions we made in the original version were quite problematic. The original exception suppression mechanism in PEP 409 (which made clever-but-idiosyncratic use of Ellipsis), was replaced with the simpler mechanism in PEP 415 (which instead uses a more conventional setter-with-side-effects-on-other-attributes approach). Reverting changes because they broke the buildbots, or because they're determined to cause bugs worse than the original problem is also not a problem - in those cases, there's new information introduced into the situation that changes the original risk assessment. An example of that was deciding to deprecate and remove contextlib.nested() before I'd figured out how to replace its legitimate use cases - I'd flat out missed a fundamental problem with the way it interacted with open() and similar resource allocation functions. The draining and demotivating cases are the ones where *no new information is introduced*, but the design decision is *challenged anyway*. Those are the ones that feel like folks are saying "We don't *actually* trust you with that authority and responsibility you've been granted, so we're going to closely police the way you use it, and expect to be able to persuade you to change your mind not through reasoned argument, but through sheer volume". For new core developers, or folks aspiring to become core developers, it is *literally* the scenario that inspired Poul-Henning Kamp to write the post at http://bikeshed.org/ (hit refresh if you get a low contrast colour scheme, it will change). Don't bother trying to improve anything, you'll just get grief and opposition for it rather than community support. For experienced core developers, it's a "Right, so you trust me to build the nuclear power plant, but not to choose the colour of the bike shed behind it" situation. Knowing we're trusted to build power plants, but will get grief for working on smaller things creates an even greater incentive to work solely on the big changes, and those are already inherently more fun to work on anyway - they really don't need the extra help. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Robert Collins writes:

On 21 July 2015 at 00:34, Ben Finney wrote:

...

Paul Moore writes:

...

Again, I'm sorry to pick on one sentence out of context, but it cut straight to my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?

That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.

Depending on what you mean by justification , this leaves no leeway for hunches, feelings, intuition, or grey area changes.

This doesn't address what I've said; or, I've misunderstood what you've said. I'm well aware that *most* of the changes we actually work on and introduce will be based on hunches, feelings, intuition, grey areas, etc. That can't be expected to disappear. Indeed, these non-rational ways of reaching a decision are essential to allow us to act with any kind of speed. Non-rational decision making is much faster, and necessarily will form the great majority of our decision making. Good! What I'm making explicit is: those can't serve as *justification* for introducing a change. When a change is challenged (by someone to whom we are answerable), claiming that it just “felt right” is not enough. I'm not proposing to block every change based on those non-rational impulses. Those impulses reflect our internalised lessons that we've learned from experience. When someone's non-rational decisions prove reliably good *when challenged* by rational third-party interrogation, we correctly trust them to make more such decisions. That's not advocating subjecting a *person* to interrogation, with the connotations of moral judgement. Be clear: I'm talking only about interrogating the rational justification for a code change, separate from judgement of the person. So: we will inevitably make most of our decisions non-rationally, which means, among other things, *faster* than explicitly rational decision making. I don't propose to change that, and our trusted core developers are not expected to be any exception to this human truth. It has to be recognised, though, that there must be some third-party scrutiny, and if no *rational* justification can be presented for a change – whether tha justification is composed ahead of the change, or some time afterward, or even at the time of the challenge – then that change is rather likely to be a poor one. Again, this is *not* a call to subject every, or even most, changes to rigorous scrutiny. That would be an intolerably slow and oppressive environment, which I don't advocate. What I am rejectiong is the idea, expressed several times in this thread, that requesting rational justification for a change can be dismissed out of hand. That's abdicating any accountability, and I was quite concerned to see that attitute expressed. No one need feel accountable to me especially, or any other arbitrary member. Rather, I think it's healthy that core developers feel accountable to the principle that any change *should* be justifiable, when challenged by someone with sufficient authority; and that such an authoritative request is not answered by “it felt right to me”.

It's also a terrible way to treat people that are contributing their own time and effort: assume good faith is a much better starting point.

Agreed, we should assume good faith. Most decisions will be reached non-rationally, and we have no option but to trust some people to make many non-rational decisions. Simultaneously, when entrusted with such decisions, we must assume that any changes we make which can't be justified *when later challenged* should fall away. What we mustn't assume is that just because we have made good non-rational decisions in the past, the next one we make will also be well-justified. Especially, we should assume that anyone who *asks* for that rational justification is also acting in good faith. -- \ “The only tyrant I accept in this world is the still voice | `\ within.” —Mohandas K. Gandhi | _o__) | Ben Finney

Paul Moore writes:

But isn't the whole *point* of a non-rational decision (as you describe it) that you *can't* articulate your reasons for making that decision.

You've conflated the process used to make a decision, with the justifications that support that decision. Those are separate. When I decide to avoid travelling a particular route to work one day, my decision could very likely be made non-rationally. Yet when challenged to say why it's a good choice, the non-rational processes I used to make the decision would rightly be dismissed. I can either present rationally compelling justification for choosing the route I used, or the challenger can rightly dismiss my choice as insufficiently justified.

You can't have your cake and eat it - are core devs allowed to make "non-rational" judgements or not?

All humans, core developers included, can and must continue to make most of our decisions in a non-rational manner. That's not the issue.

(In your view - in mine, they clearly are, and being required to justify those decisions after the fact is *not* acceptable).

I'm unsure what the objection is. The challenge is not to the person to justify themselves; it is to the change made, and what justification there is for it. This distinction is subtle but crucial. What do you imagine is the alternative? That changes which lack rational support should be allowed merely because of the person who made them? People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that. Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack. Challenging ideas can be misunderstood as challenging a person's character. That's a mistaken conflation, and I try hard to avoid it; I'd love for all of us to do the same. -- \ “If you don't know what your program is supposed to do, you'd | `\ better not start writing it.” —Edsger W. Dijkstra | _o__) | Ben Finney

Paul Moore writes:

On 28 July 2015 at 13:35, Ben Finney wrote:

...

People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that.

Good.

...

Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack.

While that is fine, you appear unwilling to accept the possibility that people may not have the time/energy to develop a detailed rational justification for a change that they have made, and demanding that they do so when they are offering the time they do give on a volunteer basis, is what I claim is unacceptable.

I've said many times now that's not what I'm advocating. I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members. The latter is what you're arguing against, and of course I agree. I've never advocated that.

The issue is not one of your motives in asking for explanations - it's the implication that you are entitled to require others to *provide* those explanations, to whatever level of detail *you* require.

Hopefully this repetition is enough: I do not claim any such entitlement.

I hope that clarifies my position.

Likewise. Thanks for engaging. -- \ “… correct code is great, code that crashes could use | `\ improvement, but incorrect code that doesn’t crash is a | _o__) horrible nightmare.” —Chris Smith, 2008-08-22 | Ben Finney

Ben Finney writes:

I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members.

The latter is what you're arguing against, and of course I agree. I've never advocated that.

Sure, but the former, when stated as a rule rather than induced from past cases, is also an unacceptably high bar. It's unnecessarily high, because this is open source. No mistake is irrecoverable, even if it happens in a public release. One can always keep using the last release one liked.<wink/> Or maintain a local fork. Or switch to a different language. Or <gasp/> live with the misfeature. The other face is that it's impossibly high. Some decisions can't be justified rationally, because the theory isn't developed until later, typically based on experience with an intuitively-approved feature. In the end, some decisions really do come down to somebody's "gut feeling". As I've already said, in the case of "assret" I *personally* think the demands of accountability were higher than the mere repetition of "it's a minor design decision" could satisfy. Nevertheless, I wouldn't try to enunciate a rule. Steve

On 07/29/2015 09:18 AM, Wes Turner wrote:

sorry, I haven't the context for this

tl;dr -- Mock objects now protect against accidentally accessing wrong assert methods by raising an attribute error; this is different than past behavior in that any attribute that didn't exist simply returned a new Mock. Besides guarding against incorrect "assert*" attributes, it also guards against "assret*" attributes. For the complete history: https://mail.python.org/pipermail/python-dev/2015-July/140720.html Please keep in mind that this has been discussed enough. -- ~Ethan~

Nick Coghlan writes:

The draining and demotivating cases are the ones where *no new information is introduced*, but the design decision is *challenged anyway*.

But this particular thread is an extreme case, that demonstrates why this kind of thing happens *despite* the good intentions of everyone involved. The design decision (to special-case a typo) that was made is a true "WTF?!" The "meta" of "special cases aren't special enough to break the rules" is that no design decision that violates it should be dismissed as "minor". In context of a mailing list, doing so is going to be taken by readers as "I know what I'm doing, and you don't know what you're talking about, so STFU." The average poster to Python-Dev surely trusts the core developers to know what they're doing. If you (as the developer or developers on the spot) don't have time to explain, at least indicate that the process implied in

Folks are granted core committer privileges because we *trust their instincts*. We trust them to know when they're acting within the limits of their own expertise and experience, and we trust them to know when it would be beneficial to seek feedback from a wider audience before making up their minds.

was followed.[1] Something like The special case of trapping the typo "assret" was discussed on python-committers. The conclusion was that in the special context of mock this is a case where "practicality beats purity". In any case, the decision has been discussed thoroughly and isn't going to change in the absence of truly new information, so further discussion here is pointless. Let's move on. would have stopped the thread in its tracks, I believe.[2] But repeated unexplained assertions that the design decision is "minor" were just a red cape to a bull. That explanation just *feels* *wrong*, it causes massive cognitive dissonance -- and makes it very hard for readers to respond appropriately to perfectly valid explanations that "this is not the time to make a change anyway".

Those are the ones that feel like folks are saying "We don't *actually* trust you with that authority and responsibility you've been granted, so we're going to closely police the way you use it, and expect to be able to persuade you to change your mind not through reasoned argument, but through sheer volume".

This is a fallacy, very human, but still the fallacy of anthropomorphizing a crowd. A very few people repeatedly posted with the content limited to "Special cases aren't", but most expressed their puzzlement in varied ways over time. Of course all of them started with "special cases aren't", which is why the proponents feel browbeaten. *Both* roles in this comedy of errors are natural, they are inherent in human cognition (citations on request), and nobody is to be blamed. At Python's scale, *some* (not all) of the core developers are going to need to recognize the "crowd" dynamic and deal with it -- or Python won't scale any further IMO. (In theory there may be better channels than mailing lists which would inhibit this dynamic, but I don't know of any practical ones.)

Don't bother trying to improve anything, you'll just get grief and opposition for it rather than community support.

For experienced core developers, it's a "Right, so you trust me to build the nuclear power plant, but not to choose the colour of the bike shed behind it" situation.

Et tu, Brutus! This point has been made *so* many times in this thread. Surely you don't think anybody missed it!<wink/> The only *practical* suggestion from "the core" has been self- restraint on the part of "the crowd". But that has severe limits *because* the "crowd" is made up of people who "aren't Dutch yet". They *are* going to ask questions, and as Python Dev gets bigger, more of them are going to ask questions, and in particularly controversial cases they are going to ask more or less simultaneously, resulting in the "browbeating effect". I think it's time to either end this "meta" thread, or refocus it from "why contributors feel sad" and "why we should trust committers" (AFAICS there is actually consensus on both, although sometimes expressed as apparent disagreement) to discussion of practical ways to resolve the conflict. That is the conflict between the need of actual contributors to get to-the-point feedback and the need of contributors- in-embryo to learn what's going on. Sadly, as long as the proposed "Communication" section in the devguide concentrates on the responsibility of commenters to exercise restraint to the exclusion of the possibility of "somebody" managing threads, it's not going to help much IMO. Python Dev is *already* as civil a forum as any I've seen, considering its scale and its success in recruiting new contributors at all levels of maturity. Steve Footnotes: [1] It occurs to me that pointing to existing discussion actually is the responsibility of an OP who "promotes" a discussion from the tracker or python-committers to python-dev. Or perhaps the proponents should have promoted to python-dev themselves. But I haven't got time to think that through. [2] N.B. Those four sentences were indeed posted several times in various combinations. But never all at once and together, and that's what's needed to have the thread-stopping effect.

On 22 July 2015 at 12:09, Paul Moore wrote:

5. Assume that the decision was well-considered and made with good reasons. If you don't understand the reasons, and feel you need to, ask for them, but refrain from judgement until you have the reasons. The original mail in this thread ("is this a joke?" is a particularly bad violation of this rule - does anyone seriously think a core dev commits code as a joke???).

My apologies - the original thread was *not* the source of the "is this a joke?" comment. That appeared somewhere else, and I can't recall exactly where. The original post was well-explained and a sensible question, which pretty much followed all of the rules I covered above (although it was still about something a year old, which might have been nice to note in the post). My apologies to Dima for misrepresenting what he said. Just goes to show, that even when going on about "do your research", I failed to do so myself :-( I suppose that emphasises the point that we all make mistakes, so let's try to remember that when discussing rules of conduct... Paul

On 23 July 2015 at 03:01, Alexander Belopolsky wrote:

On Wed, Jul 22, 2015 at 7:09 AM, Paul Moore wrote:

...

does anyone seriously think a core dev commits code as a joke???

Yes, https://hg.python.org/cpython/rev/0530aadff696. :-)

Second person to pick me up on that :-) Paul

Ben Finney writes:

Definitely agreed, and I'm not implying otherwise.

There is a distinction to be drawn:

* If challenged to do so, could one (the contributor) present a compelling justification for the change?

Aside from Paul's disclaimer, this is way too high a bar. Nick tried to express that in his post, but I doubt it's possible to successfully communicate what Nick means in a mailing list post because it's a very subtle distinction that requires a lot of "what could he be thinking" effort from *each* reader to convey.

* If a participant on this forum feels entitled to challenge a change, must the contributor present an arbitrary quantity of justification of each decision when challenged?

This expression is improperly restrictive. Contributors object to it because it's objectively objectionable, and easy to express. But they also don't want to wade through arbitrary amounts of feedback which is often context-free and/or actually misguided. They don't want to explain a point multiple times to casual readers who miss it, or spend the inordinate amount of effort required to compose a post that covers all the context needed in terms that all the casual readers can parse. All that is much harder to explain in terms which will result in *well-informed, to-the-point* feedback, and without sounding like "I know what I'm doing so STFU" (the recent "to python-dev from python-dev" post is an example of the kind of wording that can be misinterpreted this way). So people object to "micro-management" as a summary of frustration, but the actual sematics are complex and subtle.

On Jul 22 2015, Nick Coghlan wrote:

On 22 July 2015 at 13:23, Nikolaus Rath wrote:

...

If it were up to me, I'd focus all the resources of the PSF on reducing this backlog - be that by hiring some core developers to work full-time on just the open bugtracker issues, or by financing development of better code review and commit infrastructure.

Ah, but the PSF can't do that without infringing on python-dev's autonomy - switching to my PSF Director's hat, while we'd certainly be prepared to help with funding a credible grant proposal for something like the Twisted technical fellowship, we wouldn't *impose* help that the core developers haven't asked for.

I don't understand. If I would hire a core developer myself to work on this (theoretically, I have no plans to do that), would that also be infringing python-dev's authority? If so, how is that different from me doing the work? If not, why is it different if the PSF decides to hire someone?

...

The current situation looks like a downward spiral to me. New contributors are frustrated and leave because they feel their contribution is not welcome, and core developers get burned out by the gigantic backlog and the interaction with frustrated patch submitters - thus further reducing the available manpower.

We actually still have a lot of paid core developer (and potential core developer) time locked up in facilitating the Python 2 -> 3 migration, as we didn't fully appreciate the extent to which Python had been adopted in the Linux ecosystem and elsewhere until folks started seeking help upgrading.

Interesting. Is this information available publically somewhere? I'm curious what exactly is being worked on. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

* Nikolaus Rath [2015-07-21 20:23:15 -0700]:

On Jul 21 2015, Nick Coghlan wrote:

...

All of this is why the chart that I believe should be worrying people is the topmost one on this page: http://bugs.python.org/issue?@template=stats

Both the number of open issues and the number of open issues with patches are steadily trending upwards. That means the bottleneck in the current process *isn't* getting patches written in the first place, it's getting them up to the appropriate standards and applied. Yet the answer to the problem isn't a simple "recruit more core developers", as the existing core developers are *also* the bottleneck in the review and mentoring process for *new* core developers.

As a random datapoint:

Early last year I wanted get involved in CPython development. In the next months I submitted and reviewed maybe 20 - 25 patches in the bug tracker. After seeing all of them languish, I stopped submitting and reviewing and just tried to get someone to look at the issues that I'd worked on. Eventually, I managed to get almost all of them committed (the last one sometime this February, after more than a year). However, it was such an uphill battle just to get someone to look at my contributions that I haven't touched the bugtracker ever since.

[...]

As another random datapoint: I have some (minor) things which I'd like to contribute to Python - and I never did. Seeing the number of open issues with patches just make me feel like it'd be a waste of time to contribute. It seems very plausible the patches will just be ignored without me putting effort in getting them noticed. I'm fine with revising things until people are happy, i.e. I don't just want to post a patch and disappear - but I don't want to fight to get any kind of response, and it looks to me like I'd have to. :-/ Florian -- http://www.the-compiler.org | me@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/