How do we tell if we're helping or hindering the core development process? (was Re: How far to go with user-friendliness)
On 20 July 2015 at 22:34, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
Again, I'm sorry to pick on one sentence out of context, but it cut straight to my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?
That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.
No, that's not how this works: if folks are thinking that being a Python user, or even a CPython core developer, means that we're entitled to micromanage core developers by demanding extensive explanations for any arbitrary commit we choose, they're thoroughly mistaken. Only Guido has that privilege, and one of the reasons he's as respected as he is is his willingness to trust the experience and expertise of others and only rarely exercise his absolute authority. Folks are granted core committer privileges because we *trust their instincts*. We trust them to know when they're acting within the limits of their own expertise and experience, and we trust them to know when it would be beneficial to seek feedback from a wider audience before making up their minds. There are *many* cases where we *don't know* up front what the right answer is, so we'll actively seek consultation, whether that's through a review request on the issue tracker, a focused python-dev discussion, a more speculative discussion on python-ideas, or engaging in the full Python Enhancement Proposal process. There are also cases where we'll decide "it seems plausible that this might be a good idea, so let's try it out and see what happens in practice rather than continuing to speculate" - only ever doing things that you're already 100% certain are a good idea is a recipe for stagnation and decline (hence the "Now is better than never" line in the Zen). A feedback cycle of a few years is a relatively short time in programming language development, so if we discover with the benefit of hindsight that something that seemed plausible really was in fact a genuinely bad idea, that's why we have a deprecation process (as well as the abilty to mark new APIs as provisional). If we want a faster feedback cycle measured in days or weeks or months rather than years, then we'll find a way to make the idea available for experimentation via PyPI or at least as a cookbook recipe or in a public source repo. But the authority and responsibility to make changes, to decide what constitutes a reasonable risk, to decide which updates are appropriate to send out to tens of millions of Python users worldwide remains *ours*. Some of those decisions will be akin to deciding to paint a bikeshed blue instead of yellow (or green, or red, or chartreuse, or ...). Others will be about mitigating the observed negative consequences of a previous design decision while retaining the positive aspects. Those kinds of design decision are still more art than science - that's why our solution to them is to attempt to recruit people with the interest, ability and time needed to make them well, and then largely leave them to it. Explaining design decisions after the fact isn't about *defending* those decisions, it's about attempting to convey useful design knowledge, in order to help build new core contributors (and also because explaining something to someone else is a good way to understand it better yourself). All of this is why the chart that I believe should be worrying people is the topmost one on this page: http://bugs.python.org/issue?@template=stats Both the number of open issues and the number of open issues with patches are steadily trending upwards. That means the bottleneck in the current process *isn't* getting patches written in the first place, it's getting them up to the appropriate standards and applied. Yet the answer to the problem isn't a simple "recruit more core developers", as the existing core developers are *also* the bottleneck in the review and mentoring process for *new* core developers. In my view, those stats are a useful tool we can use to ask ourselves "Am I actually helping with this contribution, or at the very least, not causing harm?": * participating in the core-mentorship list, and actively looking for ways to improve the effectiveness of the mentorship program (that don't just queue up more work behind the existing core developer bottleneck) should help make those stats better * sponsoring or volunteering to help with the upstream projects our core workflow tools are based on (Roundup, Buildbot) or with updating our specific installations of them should help make those stats better (and also offers faster iteration cycles than core development itself) * helping core developers that have time to work on "CPython in general" rather than specific projects of interest to them to focus their attention more effectively may help make those stats better (and it would be even better if we could credit such triaging efforts appropriately) * exploring ways to extract contribution metrics from Roundup so we can have a more reliable detection mechanism for active and consistent contributors than the "Mark 1 core developer memory, aka the notoriously unreliable human brain" may help make those stats better * helping out the upstream projects for one or both of the forge.python.org proposals (Kallithea, Phabricator) that are designed (at least in part) to remove core developers from the critical path for management of the support repos like the developer guide and Python Enhancement Proposals may help make those stats better * getting more core contributors into a position where at least some of their work time can be spent on facilitating upstream contributions may help make those stats better (shifting at least some core contribution activity to paid roles is also one of the keys to countering the "free time for volunteer activities is inequitably distributed" problem) * alienating veteran core developers and encouraging them to spend their time elsewhere will contribute to making those stats worse * discouraging newly minted core developers from exercising their granted authority will contribute to making those stats worse * discouraging veteran core developers from recruiting new core developers by contributing to creating a hostile environment on the core mailing lists will *definitely* contribute to making those stats worse Make no mistake, sustainable open source development is a *genuinely hard problem*. We're in a much better position than most projects (being one of the top 5 programming languages in the world has its benefits), but we're still effectively running a giant multinational collaborative software development project with close to zero formal management support. While their are good examples we can (and are) learning from, improving the support structures for an already wildly successful open source project without alienating existing contributors isn't a task that comes with an instruction manual :) Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hello, since this thread is restarting in debriefing mode: one thing struck me as a non-committer following python-dev. It seems that we (non-committers) have a difficulty making the distinction between pre-implementation design discussions (PEPs beeing the typical example), where relevant technical comments are welcome, even on minor points, and post-commit discussions, where the maintainer already made the design decisions, and discussing anything except real bugs is unhelpful. In this particular thread, the distinction was all the more blurred by the very generic title. A thread titled something like "Commit-xxx: should mock try to detect user typos?" would have been much less likely to attract comments discussing principles in the abstract, without taking the context into account. Regards, Baptiste
Nick Coghlan <ncoghlan@gmail.com> writes:
On 20 July 2015 at 22:34, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
[…] my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?
That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.
No, that's not how this works: if folks are thinking that being a Python user, or even a CPython core developer, means that we're entitled to micromanage core developers by demanding extensive explanations for any arbitrary commit we choose, they're thoroughly mistaken.
Definitely agreed, and I'm not implying otherwise. There is a distinction to be drawn: * If challenged to do so, could one (the contributor) present a compelling justification for the change? This is what I claim Paul Moore's doubt (fear?) is indicative of. I maintain that this doubt is quite healthy: it helps the contributor to pause, reflect, seek assistance in making decisions, and thereby also tends to exclude poorly-justified changes which would otherwise be committed. * If a participant on this forum feels entitled to challenge a change, must the contributor present an arbitrary quantity of justification of each decision when challenged? This seems to be what many contributors object to, but I don't assert this at all. I never meant to imply that mere participation here entitles a person to demand justification for changes.
Folks are granted core committer privileges because we *trust their instincts*. We trust them to know when they're acting within the limits of their own expertise and experience, and we trust them to know when it would be beneficial to seek feedback from a wider audience before making up their minds.
That's all good, and it's quite compatible with the healthy doubt I described above.
But the authority and responsibility to make changes, to decide what constitutes a reasonable risk, to decide which updates are appropriate to send out to tens of millions of Python users worldwide remains *ours*.
No objection from me. I am not asking that contributors be at the mercy of limitless demands for justification from arbitrary participants in this forum. I am asking that contributors should *have* compelling justification for any change they commit, if someone with sufficient authority were to ask for it. That doesn't make contributors any more answerable to arbitrary participants here. It does make explicit that contributors, while trusted of course, are answerable to more than their instincts and what “feels right”: they are answerable also the community of core contributors as a condition of that trust. They are also effectively answerable to their own conscience about what justification that community may ask in future. -- \ “We suffer primarily not from our vices or our weaknesses, but | `\ from our illusions.” —Daniel J. Boorstin, historian, 1914–2004 | _o__) | Ben Finney
On 21 July 2015 at 11:03, Ben Finney <ben+python@benfinney.id.au> wrote:
* If challenged to do so, could one (the contributor) present a compelling justification for the change?
This is what I claim Paul Moore's doubt (fear?) is indicative of. I maintain that this doubt is quite healthy: it helps the contributor to pause, reflect, seek assistance in making decisions, and thereby also tends to exclude poorly-justified changes which would otherwise be committed.
That is *not* what I was trying to express. My fear is that I will be subjected to the sort of unreasonable level of debate and frankly criticism that came up in this thread, and I'm not sure if I have the energy to deal with it. I wouldn't ever commit something unless *in my judgement* it was OK to go in. Whether my judgement is sufficient is the whole point here. And to be honest, there's an implication in your comment that you think there is a possibility that any of the core devs might commit something when they had a level of doubt[1] about whether it was right to do so. I think that implication is unwarranted, and constitutes exactly the sort of subtle criticism that escalates things like this. I'm sure that's not what you intended, and I *know* I've reached a point with this thread where I'm over-sensitive to such implications, but it's a good illustration of how even a well-meaning comment can be perceived very differently than it was intended by the recipient. For something which is a hobby for me, I'd rather feel more joy in what I achieve, and less burden of responsibility. Changing code that affects I-don't-even-want-to-know-how-many people and businesses is quite enough responsibility without also having to put up with only hearing from people who disagree with what I do, and never from people who are grateful... End of unnecessarily emotional outpouring, we now return you to your regular scheduled flamewars :-) Paul [1] That's doubt, not mistakes. People make mistakes. The big problem here is python-dev is becoming pretty hostile to people who (are perceived to) make mistakes. That is, ultimately, all of us.
On 21 July 2015 at 21:19, Paul Moore <p.f.moore@gmail.com> wrote:
On 21 July 2015 at 11:03, Ben Finney <ben+python@benfinney.id.au> wrote:
* If challenged to do so, could one (the contributor) present a compelling justification for the change?
This is what I claim Paul Moore's doubt (fear?) is indicative of. I maintain that this doubt is quite healthy: it helps the contributor to pause, reflect, seek assistance in making decisions, and thereby also tends to exclude poorly-justified changes which would otherwise be committed.
That is *not* what I was trying to express. My fear is that I will be subjected to the sort of unreasonable level of debate and frankly criticism that came up in this thread, and I'm not sure if I have the energy to deal with it. I wouldn't ever commit something unless *in my judgement* it was OK to go in. Whether my judgement is sufficient is the whole point here.
Exactly. Being granted a commit bit means "The folks that got CPython to where it is today trust you to take appropriate risks in guiding CPython's future". It can be safely assumed that *everyone* that has that authority takes it *very* seriously (otherwise they wouldn't have been granted commit privileges). If there weren't any risks to be weighed and judgment calls to be made, we could let a computer decide what commits should go in (and indeed "make patchcheck" already handles some of those basic tasks, and we'd like to automate even more of them like "does this patch apply, and do the tests still pass on x86_64 under at least Windows and a long term support Linux distro after applying it?"). This challenge is reflected in the fact that the Zen of Python is deliberately self-contradictory, as it articulates competing design principles to take into consideration, rather than being able to provide ironclad rules that avoid the need for human judgement in determining *which* of those design guidelines are most salient in any given situation. Now, proposing further *enhancements* to a change is perfectly reasonable, especially if the feedback is based on practical experience with the original version of the change. The implementation of PEP 492 (async/await), for example, underwent some fundamental changes during the beta process, as developers' feedback showed that some of the design decisions we made in the original version were quite problematic. The original exception suppression mechanism in PEP 409 (which made clever-but-idiosyncratic use of Ellipsis), was replaced with the simpler mechanism in PEP 415 (which instead uses a more conventional setter-with-side-effects-on-other-attributes approach). Reverting changes because they broke the buildbots, or because they're determined to cause bugs worse than the original problem is also not a problem - in those cases, there's new information introduced into the situation that changes the original risk assessment. An example of that was deciding to deprecate and remove contextlib.nested() before I'd figured out how to replace its legitimate use cases - I'd flat out missed a fundamental problem with the way it interacted with open() and similar resource allocation functions. The draining and demotivating cases are the ones where *no new information is introduced*, but the design decision is *challenged anyway*. Those are the ones that feel like folks are saying "We don't *actually* trust you with that authority and responsibility you've been granted, so we're going to closely police the way you use it, and expect to be able to persuade you to change your mind not through reasoned argument, but through sheer volume". For new core developers, or folks aspiring to become core developers, it is *literally* the scenario that inspired Poul-Henning Kamp to write the post at http://bikeshed.org/ (hit refresh if you get a low contrast colour scheme, it will change). Don't bother trying to improve anything, you'll just get grief and opposition for it rather than community support. For experienced core developers, it's a "Right, so you trust me to build the nuclear power plant, but not to choose the colour of the bike shed behind it" situation. Knowing we're trusted to build power plants, but will get grief for working on smaller things creates an even greater incentive to work solely on the big changes, and those are already inherently more fun to work on anyway - they really don't need the extra help. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 21 July 2015 at 00:34, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
Again, I'm sorry to pick on one sentence out of context, but it cut straight to my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?
That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.
Depending on what you mean by justification , this leaves no leeway for hunches, feelings, intuition, or grey area changes. It's also a terrible way to treat people that are contributing their own time and effort: assume good faith is a much better starting point. I think its reasonable to say that any change should be open to post hoc discussion. Thats how we learn, but justification and challenging is an adversarial framing, and one I'm flatly uninterested in. Life is too short to volunteer on adversarial activies.
That isn't a mandate to challenge every change, of course. It does mean that every change should be considered in light of “Can I justify this, if challenged?”
So what you describe sounds like a healthy barrier: one which works to keep out unjustifiable changes.
I don't understand why thats healthy or useful.
What is needed is to have both that *and* the support of the community so it's not a barrier to the *contributors*. The contributors should not feel excluded merely because some of their changes might need to be.
I don't think that contributors are a problem here. But, I'm going to dig into that more in reply to Nick.
Hmm, maybe I'd better hold off and let someone else make the decision...
What of the (obvious, to me) option to retain the authority to make the decision, but take the doubt as a sign that one should consult with others before making the decision?
That is, there's no need to feel that one shouldn't make the decision. But perhaps one shouldn't make it solely on one's own experience or insight. Get others involved, even non-committers, and discuss it, and understand the issue better. With that improved basis, then make the decision.
As a case study, this discussion where something like 90% of the kibbitzing demonstrated a clear lack of understanding of the very behaviour of Mock, *and* the change in question was discussed, *prior* to it being done, *and* users have welcomed it....I must conclude that either: - this discussion was the exception to prove your general rule (but why would we derive that general rule from this discussion) - the general rule isn't general.
Am I naive to think that's desirable for PYthon core committers?
What's the goal here: what actual problem are we trying to solve for? More contributors? A better Python more people can use? Keeping up with the contributions we've already received but not actioned? [...] Like: pick one thing. What we /really/ want to achieve, then lets look at what will let us get there. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
Robert Collins <robertc@robertcollins.net> writes:
On 21 July 2015 at 00:34, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
Again, I'm sorry to pick on one sentence out of context, but it cut straight to my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?
That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.
Depending on what you mean by justification , this leaves no leeway for hunches, feelings, intuition, or grey area changes.
This doesn't address what I've said; or, I've misunderstood what you've said. I'm well aware that *most* of the changes we actually work on and introduce will be based on hunches, feelings, intuition, grey areas, etc. That can't be expected to disappear. Indeed, these non-rational ways of reaching a decision are essential to allow us to act with any kind of speed. Non-rational decision making is much faster, and necessarily will form the great majority of our decision making. Good! What I'm making explicit is: those can't serve as *justification* for introducing a change. When a change is challenged (by someone to whom we are answerable), claiming that it just “felt right” is not enough. I'm not proposing to block every change based on those non-rational impulses. Those impulses reflect our internalised lessons that we've learned from experience. When someone's non-rational decisions prove reliably good *when challenged* by rational third-party interrogation, we correctly trust them to make more such decisions. That's not advocating subjecting a *person* to interrogation, with the connotations of moral judgement. Be clear: I'm talking only about interrogating the rational justification for a code change, separate from judgement of the person. So: we will inevitably make most of our decisions non-rationally, which means, among other things, *faster* than explicitly rational decision making. I don't propose to change that, and our trusted core developers are not expected to be any exception to this human truth. It has to be recognised, though, that there must be some third-party scrutiny, and if no *rational* justification can be presented for a change – whether tha justification is composed ahead of the change, or some time afterward, or even at the time of the challenge – then that change is rather likely to be a poor one. Again, this is *not* a call to subject every, or even most, changes to rigorous scrutiny. That would be an intolerably slow and oppressive environment, which I don't advocate. What I am rejectiong is the idea, expressed several times in this thread, that requesting rational justification for a change can be dismissed out of hand. That's abdicating any accountability, and I was quite concerned to see that attitute expressed. No one need feel accountable to me especially, or any other arbitrary member. Rather, I think it's healthy that core developers feel accountable to the principle that any change *should* be justifiable, when challenged by someone with sufficient authority; and that such an authoritative request is not answered by “it felt right to me”.
It's also a terrible way to treat people that are contributing their own time and effort: assume good faith is a much better starting point.
Agreed, we should assume good faith. Most decisions will be reached non-rationally, and we have no option but to trust some people to make many non-rational decisions. Simultaneously, when entrusted with such decisions, we must assume that any changes we make which can't be justified *when later challenged* should fall away. What we mustn't assume is that just because we have made good non-rational decisions in the past, the next one we make will also be well-justified. Especially, we should assume that anyone who *asks* for that rational justification is also acting in good faith. -- \ “The only tyrant I accept in this world is the still voice | `\ within.” —Mohandas K. Gandhi | _o__) | Ben Finney
On 28 July 2015 at 05:18, Ben Finney <ben+python@benfinney.id.au> wrote:
Indeed, these non-rational ways of reaching a decision are essential to allow us to act with any kind of speed. Non-rational decision making is much faster, and necessarily will form the great majority of our decision making. Good!
What I'm making explicit is: those can't serve as *justification* for introducing a change. When a change is challenged (by someone to whom we are answerable), claiming that it just “felt right” is not enough.
But isn't the whole *point* of a non-rational decision (as you describe it) that you *can't* articulate your reasons for making that decision. You can't have your cake and eat it - are core devs allowed to make "non-rational" judgements or not? (In your view - in mine, they clearly are, and being required to justify those decisions after the fact is *not* acceptable). Paul
Paul Moore <p.f.moore@gmail.com> writes:
But isn't the whole *point* of a non-rational decision (as you describe it) that you *can't* articulate your reasons for making that decision.
You've conflated the process used to make a decision, with the justifications that support that decision. Those are separate. When I decide to avoid travelling a particular route to work one day, my decision could very likely be made non-rationally. Yet when challenged to say why it's a good choice, the non-rational processes I used to make the decision would rightly be dismissed. I can either present rationally compelling justification for choosing the route I used, or the challenger can rightly dismiss my choice as insufficiently justified.
You can't have your cake and eat it - are core devs allowed to make "non-rational" judgements or not?
All humans, core developers included, can and must continue to make most of our decisions in a non-rational manner. That's not the issue.
(In your view - in mine, they clearly are, and being required to justify those decisions after the fact is *not* acceptable).
I'm unsure what the objection is. The challenge is not to the person to justify themselves; it is to the change made, and what justification there is for it. This distinction is subtle but crucial. What do you imagine is the alternative? That changes which lack rational support should be allowed merely because of the person who made them? People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that. Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack. Challenging ideas can be misunderstood as challenging a person's character. That's a mistaken conflation, and I try hard to avoid it; I'd love for all of us to do the same. -- \ “If you don't know what your program is supposed to do, you'd | `\ better not start writing it.” —Edsger W. Dijkstra | _o__) | Ben Finney
On 28 July 2015 at 13:35, Ben Finney <ben+python@benfinney.id.au> wrote:
People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that.
Good.
Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack.
While that is fine, you appear unwilling to accept the possibility that people may not have the time/energy to develop a detailed rational justification for a change that they have made, and demanding that they do so when they are offering the time they do give on a volunteer basis, is what I claim is unacceptable.
Those are separate. When I decide to avoid travelling a particular route to work one day, my decision could very likely be made non-rationally.
Would you be happy if someone insisted, when you arrive at work, that you rigorously justify your choice, down to the smallest detail? "Why didn't you avoid road A, as there's a bridge over that road and when you go that way you risk the bridge collapsing and killing you? There's no way it makes sense to take such a significant risk when by going down road B you could avoid it, for no more than 10 minutes' delay". As a further point, even if your challenge is from a desire to ensure the best possible outcome for Python, you may not know the subject area as well as the core dev involved (that's pretty much by definition for these mailing list debates). In order to explain the decision, the core dev may need to give you a lot of information that they already know, from experience. Your demand for explanation has now turned into a demand for free education. Is that still acceptable? Must Steve Dower or I explain all of the relevant intricacies of Windows to you just so that you can understand our explanation of why we made a particular Windows-related change? The issue is not one of your motives in asking for explanations - it's the implication that you are entitled to require others to *provide* those explanations, to whatever level of detail *you* require. I hope that clarifies my position. In the spirit of what I've said above, I hope you won't take it the wrong way if I point out that this discussion has become a drain on my limited personal time, and I am no longer finding it of interest. As a result, I'm not going to follow up further. If what I've said doesn't justify my position sufficiently, we'll simply have to agree to differ. Paul
Paul Moore <p.f.moore@gmail.com> writes:
On 28 July 2015 at 13:35, Ben Finney <ben+python@benfinney.id.au> wrote:
People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that.
Good.
Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack.
While that is fine, you appear unwilling to accept the possibility that people may not have the time/energy to develop a detailed rational justification for a change that they have made, and demanding that they do so when they are offering the time they do give on a volunteer basis, is what I claim is unacceptable.
I've said many times now that's not what I'm advocating. I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members. The latter is what you're arguing against, and of course I agree. I've never advocated that.
The issue is not one of your motives in asking for explanations - it's the implication that you are entitled to require others to *provide* those explanations, to whatever level of detail *you* require.
Hopefully this repetition is enough: I do not claim any such entitlement.
I hope that clarifies my position.
Likewise. Thanks for engaging. -- \ “… correct code is great, code that crashes could use | `\ improvement, but incorrect code that doesn’t crash is a | _o__) horrible nightmare.” —Chris Smith, 2008-08-22 | Ben Finney
On 29 July 2015 at 00:17, Ben Finney <ben+python@benfinney.id.au> wrote:
I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members.
The latter is what you're arguing against, and of course I agree. I've never advocated that.
The thing we work toward as core developers, and aspiring core developers, is good design intuition that aligns with the Zen of Python. That doesn't mean we're all going to be able to perfectly *articulate* that intuition in every case, and even when we can, those explanations are themselves generally going to be backed by intuition rather than rigorous scientific research. Perhaps we need a new initialism to summarise these kinds of cases in a way that isn't dismissive of the concerns of folks that still have doubts about a decision: LTIASHIG (Let's Try It And See How It Goes) It's important to remember that *having* good design intuition is a *separate* skill from *explaining* design intuition well. The former involves being able to bring personal experience to bear on a design task, and produce a result that a non-trivial subset of people enjoy using. The latter involves things like learning a lot more about human cognition, and why certain concepts cause the human brain to get jammed up, and how the way a programming language models a particular domain relates to how folks learn about that domain in the first place. One of the best engineers I ever worked for (my first full-time boss) had *brilliant* design instincts, but was substantially weaker when it came to explaining the rationale for his decisions. Spending a few years figuring out why the software architecture he designed was such a good one turned out to be one of the most valuable learning experiences of my life (and, not coincidentally, was my original exposure to the "communicating sequential processes" approach to concurrency and parallelism). CPython core developers have *real authority* as *individuals* over a language ecosystem used by millions of people. That's a large part of why we're relatively cautious about handing out commit bits - it's not just about the technical aspects, it's about developers' design judgment, and intuitions on when things should be escalated for wider discussion *before* making a decision. Sometimes that intuition is going to tell us "this change is worth the risk, even though I can't fully explain why I think that, and hearing further opinions without supporting evidence isn't going to change my mind" . Having someone willing to make that kind of call is one of the big reasons modules with an active lead maintainer can more readily evolve than the collectively maintained modules that don't have an authoritative voice guiding their API design. And when we make genuine design mistakes (as we inevitably will)? That's one of the things the deprecation process is for: if a mistake is actively causing problems for folks attempting to use a feature, then we have options to deal with it, rather than having to carry it forever. In many cases, we can even go with a better approach, where changing our minds later involves making a *backwards compatible API change*, reducing the practical long term design risk of postponing a particular aspect of a decision to near zero. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Ben Finney writes:
I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members.
The latter is what you're arguing against, and of course I agree. I've never advocated that.
Sure, but the former, when stated as a rule rather than induced from past cases, is also an unacceptably high bar. It's unnecessarily high, because this is open source. No mistake is irrecoverable, even if it happens in a public release. One can always keep using the last release one liked.<wink/> Or maintain a local fork. Or switch to a different language. Or <gasp/> live with the misfeature. The other face is that it's impossibly high. Some decisions can't be justified rationally, because the theory isn't developed until later, typically based on experience with an intuitively-approved feature. In the end, some decisions really do come down to somebody's "gut feeling". As I've already said, in the case of "assret" I *personally* think the demands of accountability were higher than the mere repetition of "it's a minor design decision" could satisfy. Nevertheless, I wouldn't try to enunciate a rule. Steve
On Jul 28, 2015 10:41 PM, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
Ben Finney writes:
I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members.
The latter is what you're arguing against, and of course I agree. I've never advocated that.
Sure, but the former, when stated as a rule rather than induced from past cases, is also an unacceptably high bar. It's unnecessarily high, because this is open source. No mistake is irrecoverable, even if it happens in a public release. One can always keep using the last release one liked.<wink/> Or maintain a local fork. Or switch to a different language. Or <gasp/> live with the misfeature.
The other face is that it's impossibly high. Some decisions can't be justified rationally, because the theory isn't developed until later, typically based on experience with an intuitively-approved feature. In the end, some decisions really do come down to somebody's "gut feeling".
As I've already said, in the case of "assret" I *personally* think the demands of accountability were higher than the mere repetition of "it's a minor design decision" could satisfy. Nevertheless, I wouldn't try to enunciate a rule.
* sorry, I haven't the context for this: would -m compileall or an AST preprocess help catch speling mistakes as well as syntax highlighting? * If the constraints are ill-defined, there are not enough tests; "Fearless Refactoring"
Steve
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
On 07/29/2015 09:18 AM, Wes Turner wrote:
sorry, I haven't the context for this
tl;dr -- Mock objects now protect against accidentally accessing wrong assert methods by raising an attribute error; this is different than past behavior in that any attribute that didn't exist simply returned a new Mock. Besides guarding against incorrect "assert*" attributes, it also guards against "assret*" attributes. For the complete history: https://mail.python.org/pipermail/python-dev/2015-July/140720.html Please keep in mind that this has been discussed enough. -- ~Ethan~
On 29 July 2015 at 02:17, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
On 28 July 2015 at 13:35, Ben Finney <ben+python@benfinney.id.au> wrote:
People can, do, and probably must make many decisions through non-rational processes. I don't propose to change that.
Good.
Choices can be made that, when challenged, lack compelling rational justification. I do propose that such a challenge should be taken as a healthy desire to improve Python, not a personal attack.
While that is fine, you appear unwilling to accept the possibility that people may not have the time/energy to develop a detailed rational justification for a change that they have made, and demanding that they do so when they are offering the time they do give on a volunteer basis, is what I claim is unacceptable.
I've said many times now that's not what I'm advocating.
I've made a clear distinction between the need to *be able to* justify a change, versus arbitrary demands to do so by arbitrary members.
The latter is what you're arguing against, and of course I agree. I've never advocated that.
I'm arguing against the former. Being able to survive a crowd sourced grilling on any arbitrary change would be quite the chilling effect, and its a level of backpressure that the committers who engaged in this discussion have rejected. Some have rejected contributing *at all* as a result of the discussion. Others, like me, are telling you that you're wrong, that we don't accept that we can be called up for any odd commit and asked to justify it to anyone. There is a social contract around our commits - and it does permit enquiry and discussion, but not with the degree of heat or antagonism that was present in this thread. AND Not by uninformed folk: If you're going to second guess the onus is on you to educate yourself about the issue first. This particular one, for instance, requires going back through the history of mock right to its founding in 2007, and walking forward through the merge into the stdlib in Python 3,3 (because its popular) and finally the realisation that large chunks of peoples code were silently not testing what was desired and the fixing of that. Discussing the thing we discussed *in that context* is a very different discussion to what we had, where every second message was someone misunderstanding what the issue is and chiming in to say that this is surprising and unPythonic and against the Zen and oh my.
The issue is not one of your motives in asking for explanations - it's the implication that you are entitled to require others to *provide* those explanations, to whatever level of detail *you* require.
Hopefully this repetition is enough: I do not claim any such entitlement.
If you don't claim such entitlement, who does? Whose entitlement are you arguing for? If its Guido's, I think we can stop arguing - sure, he is entitled to ask for a lot, but I don't want to argue about what entitlements someone else has: they can argue on their own. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
Nick Coghlan writes:
The draining and demotivating cases are the ones where *no new information is introduced*, but the design decision is *challenged anyway*.
But this particular thread is an extreme case, that demonstrates why this kind of thing happens *despite* the good intentions of everyone involved. The design decision (to special-case a typo) that was made is a true "WTF?!" The "meta" of "special cases aren't special enough to break the rules" is that no design decision that violates it should be dismissed as "minor". In context of a mailing list, doing so is going to be taken by readers as "I know what I'm doing, and you don't know what you're talking about, so STFU." The average poster to Python-Dev surely trusts the core developers to know what they're doing. If you (as the developer or developers on the spot) don't have time to explain, at least indicate that the process implied in
Folks are granted core committer privileges because we *trust their instincts*. We trust them to know when they're acting within the limits of their own expertise and experience, and we trust them to know when it would be beneficial to seek feedback from a wider audience before making up their minds.
was followed.[1] Something like The special case of trapping the typo "assret" was discussed on python-committers. The conclusion was that in the special context of mock this is a case where "practicality beats purity". In any case, the decision has been discussed thoroughly and isn't going to change in the absence of truly new information, so further discussion here is pointless. Let's move on. would have stopped the thread in its tracks, I believe.[2] But repeated unexplained assertions that the design decision is "minor" were just a red cape to a bull. That explanation just *feels* *wrong*, it causes massive cognitive dissonance -- and makes it very hard for readers to respond appropriately to perfectly valid explanations that "this is not the time to make a change anyway".
Those are the ones that feel like folks are saying "We don't *actually* trust you with that authority and responsibility you've been granted, so we're going to closely police the way you use it, and expect to be able to persuade you to change your mind not through reasoned argument, but through sheer volume".
This is a fallacy, very human, but still the fallacy of anthropomorphizing a crowd. A very few people repeatedly posted with the content limited to "Special cases aren't", but most expressed their puzzlement in varied ways over time. Of course all of them started with "special cases aren't", which is why the proponents feel browbeaten. *Both* roles in this comedy of errors are natural, they are inherent in human cognition (citations on request), and nobody is to be blamed. At Python's scale, *some* (not all) of the core developers are going to need to recognize the "crowd" dynamic and deal with it -- or Python won't scale any further IMO. (In theory there may be better channels than mailing lists which would inhibit this dynamic, but I don't know of any practical ones.)
Don't bother trying to improve anything, you'll just get grief and opposition for it rather than community support.
For experienced core developers, it's a "Right, so you trust me to build the nuclear power plant, but not to choose the colour of the bike shed behind it" situation.
Et tu, Brutus! This point has been made *so* many times in this thread. Surely you don't think anybody missed it!<wink/> The only *practical* suggestion from "the core" has been self- restraint on the part of "the crowd". But that has severe limits *because* the "crowd" is made up of people who "aren't Dutch yet". They *are* going to ask questions, and as Python Dev gets bigger, more of them are going to ask questions, and in particularly controversial cases they are going to ask more or less simultaneously, resulting in the "browbeating effect". I think it's time to either end this "meta" thread, or refocus it from "why contributors feel sad" and "why we should trust committers" (AFAICS there is actually consensus on both, although sometimes expressed as apparent disagreement) to discussion of practical ways to resolve the conflict. That is the conflict between the need of actual contributors to get to-the-point feedback and the need of contributors- in-embryo to learn what's going on. Sadly, as long as the proposed "Communication" section in the devguide concentrates on the responsibility of commenters to exercise restraint to the exclusion of the possibility of "somebody" managing threads, it's not going to help much IMO. Python Dev is *already* as civil a forum as any I've seen, considering its scale and its success in recruiting new contributors at all levels of maturity. Steve Footnotes: [1] It occurs to me that pointing to existing discussion actually is the responsibility of an OP who "promotes" a discussion from the tracker or python-committers to python-dev. Or perhaps the proponents should have promoted to python-dev themselves. But I haven't got time to think that through. [2] N.B. Those four sentences were indeed posted several times in various combinations. But never all at once and together, and that's what's needed to have the thread-stopping effect.
On 22 July 2015 at 03:18, Stephen J. Turnbull <stephen@xemacs.org> wrote:
The only *practical* suggestion from "the core" has been self- restraint on the part of "the crowd"
I would have said the following has been covered, but maybe not. At the risk of repeating something that's already been said, here are some guidelines for people asking for comments on a commit. 1. Post *once*. Don't post again until there is some feedback from the core dev addressed - the reason threads become self-sustaining is that people respond to each other without any new input from the "other side" (i.e. the core dev who made the commit/decision). 2. Do your research - read the issue tracker comments before posting. Look at dates, times, and history. The commit that triggered this thread was made over a year ago - we're discussing ancient history here - and the original issue was raised by the lead developer on mock. There's not a lot of explanation in there, so *maybe* asking for more detail is acceptable, but what's wrong with a comment on the issue along the lines of "I know this is history, but are there links to any discussion on including the assret mis-spelling that I could read up on?") 3. Prefer comments on the tracker over mailing list discussions. Consider hard why you feel it's necessary to take the discussion to a forum with a wider readership. Honestly assess whether or not you're simply hoping to muster numbers in support of your view. If your arguments are no better than "look, lots of people agree with me", then leave it to someone with more compelling arguments (if there are any) to speak up. 4. Seriously consider, if you don't use the functionality in question, why do you feel entitled to an opinion? Bystanders' views are not disallowed, but you should start from the assumption that the opinions of people directly affected by the change have *far* more weight than yours. 5. Assume that the decision was well-considered and made with good reasons. If you don't understand the reasons, and feel you need to, ask for them, but refrain from judgement until you have the reasons. The original mail in this thread ("is this a joke?" is a particularly bad violation of this rule - does anyone seriously think a core dev commits code as a joke???). Yes, these constitute a very high bar for commenters. So what? The bars being set by commenters on core developers are also very high. Paul
On 22 July 2015 at 12:09, Paul Moore <p.f.moore@gmail.com> wrote:
5. Assume that the decision was well-considered and made with good reasons. If you don't understand the reasons, and feel you need to, ask for them, but refrain from judgement until you have the reasons. The original mail in this thread ("is this a joke?" is a particularly bad violation of this rule - does anyone seriously think a core dev commits code as a joke???).
My apologies - the original thread was *not* the source of the "is this a joke?" comment. That appeared somewhere else, and I can't recall exactly where. The original post was well-explained and a sensible question, which pretty much followed all of the rules I covered above (although it was still about something a year old, which might have been nice to note in the post). My apologies to Dima for misrepresenting what he said. Just goes to show, that even when going on about "do your research", I failed to do so myself :-( I suppose that emphasises the point that we all make mistakes, so let's try to remember that when discussing rules of conduct... Paul
On Wed, Jul 22, 2015 at 7:09 AM, Paul Moore <p.f.moore@gmail.com> wrote:
does anyone seriously think a core dev commits code as a joke???
On 23 July 2015 at 03:01, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
On Wed, Jul 22, 2015 at 7:09 AM, Paul Moore <p.f.moore@gmail.com> wrote:
does anyone seriously think a core dev commits code as a joke???
Second person to pick me up on that :-) Paul
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
[…] The "meta" of "special cases aren't special enough to break the rules" is that no design decision that violates it should be dismissed as "minor".
Thank you. That dismissal was very upsetting; essentially telling Python users that their concerns for a clean API in the standard library aren't worth much to the Python core developers.
In context of a mailing list, doing so is going to be taken by readers as "I know what I'm doing, and you don't know what you're talking about, so STFU."
That may not have been the intent. It certainly was how it was received by some of us here.
*Both* roles in this comedy of errors are natural, they are inherent in human cognition (citations on request), and nobody is to be blamed.
Since it can't seem to be said enough, I agree with what Stephen's saying here wholeheartedly: the above explications are not intended as blame, but an explanation of why calls to “stop talking about this, it's minor” had precisely the opposite effect. -- \ “Remember: every member of your ‘target audience’ also owns a | `\ broadcasting station. These ‘targets’ can shoot back.” —Michael | _o__) Rathbun to advertisers, news.admin.net-abuse.email | Ben Finney
Ben Finney writes:
Definitely agreed, and I'm not implying otherwise.
There is a distinction to be drawn:
* If challenged to do so, could one (the contributor) present a compelling justification for the change?
Aside from Paul's disclaimer, this is way too high a bar. Nick tried to express that in his post, but I doubt it's possible to successfully communicate what Nick means in a mailing list post because it's a very subtle distinction that requires a lot of "what could he be thinking" effort from *each* reader to convey.
* If a participant on this forum feels entitled to challenge a change, must the contributor present an arbitrary quantity of justification of each decision when challenged?
This expression is improperly restrictive. Contributors object to it because it's objectively objectionable, and easy to express. But they also don't want to wade through arbitrary amounts of feedback which is often context-free and/or actually misguided. They don't want to explain a point multiple times to casual readers who miss it, or spend the inordinate amount of effort required to compose a post that covers all the context needed in terms that all the casual readers can parse. All that is much harder to explain in terms which will result in *well-informed, to-the-point* feedback, and without sounding like "I know what I'm doing so STFU" (the recent "to python-dev from python-dev" post is an example of the kind of wording that can be misinterpreted this way). So people object to "micro-management" as a summary of frustration, but the actual sematics are complex and subtle.
On 21 July 2015 at 19:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 20 July 2015 at 22:34, Ben Finney <ben+python@benfinney.id.au> wrote:
Paul Moore <p.f.moore@gmail.com> writes:
Again, I'm sorry to pick on one sentence out of context, but it cut straight to my biggest fear when doing a commit (on any project) - what if, after all the worrying and consideration I put into doing this commit, people disagree with me (or worse still, I made a mistake)? Will I be able to justify what I decided?
That seems quite healthy to me. On a collaborative project with effects far beyond oneself, yes, any change *should* be able to be justified when challenged.
No, that's not how this works: if folks are thinking that being a Python user, or even a CPython core developer, means that we're entitled to micromanage core developers by demanding extensive explanations for any arbitrary commit we choose, they're thoroughly mistaken. Only Guido has that privilege, and one of the reasons he's as respected as he is is his willingness to trust the experience and expertise of others and only rarely exercise his absolute authority.
I wouldn't even agree that Guido is entitled to micromanage us. He is certainly entitled to demand explanations etc if he feels appropriate - but if that turned into micromanaging, I think we'd be in a very bad place.
All of this is why the chart that I believe should be worrying people is the topmost one on this page: http://bugs.python.org/issue?@template=stats
Both the number of open issues and the number of open issues with patches are steadily trending upwards. That means the bottleneck in the current process *isn't* getting patches written in the first place, it's getting them up to the appropriate standards and applied.
Perhaps.
Yet the answer to the problem isn't a simple "recruit more core developers", as the existing core developers are *also* the bottleneck in the review and mentoring process for *new* core developers.
sidebar: So, anyone here familiar with theory of constraints - e.g. 'The Goal', 'Critical Chain' etc? Might be interesting to do some brainstorming in that shape if folk are interest. Having open idle patches is I think generally acknowledged as a poor thing. I think of them as inventory in a manufacturing plant: they take up space, they take effort to create, they offer no value [until they're actually shipped to users], and they have direct negatives (tracker is harder to work with due to the signal-to-noise ratio, perception of the project suffers, contributors make the rational discussion not to contribute further...). Lets assume that our actual goal is to ship new Python versions, offering more and better things to users. AFAIK we're not lacking any infrastructure resources - we have enough buildbots, we have freely available compilers for all platforms.
In my view, those stats are a useful tool we can use to ask ourselves "Am I actually helping with this contribution, or at the very least, not causing harm?":
I like this approach :). But - can we go further, can we actively protect core committer time such that they waste less of it? Adding core committers won't help if the problem isn't the number of committers, but rather the amount of the time that they can devote to Python that actually gets spent on committer-activities.
* helping core developers that have time to work on "CPython in general" rather than specific projects of interest to them to focus their attention more effectively may help make those stats better (and it would be even better if we could credit such triaging efforts appropriately)
Iterating on a patch someone else put up can help. Making sure its passing tests, trying it with ecosystem projects and giving feedback.
* exploring ways to extract contribution metrics from Roundup so we can have a more reliable detection mechanism for active and consistent contributors than the "Mark 1 core developer memory, aka the notoriously unreliable human brain" may help make those stats better
OTOH, make sure that what we measure provokes things that help our goal :). Double edged sword this.
Make no mistake, sustainable open source development is a *genuinely hard problem*. We're in a much better position than most projects (being one of the top 5 programming languages in the world has its benefits), but we're still effectively running a giant multinational collaborative software development project with close to zero formal management support. While their are good examples we can (and are) learning from, improving the support structures for an already wildly successful open source project without alienating existing contributors isn't a task that comes with an instruction manual :)
+1 -Rob
On Jul 21 2015, Nick Coghlan <ncoghlan@gmail.com> wrote:
All of this is why the chart that I believe should be worrying people is the topmost one on this page: http://bugs.python.org/issue?@template=stats
Both the number of open issues and the number of open issues with patches are steadily trending upwards. That means the bottleneck in the current process *isn't* getting patches written in the first place, it's getting them up to the appropriate standards and applied. Yet the answer to the problem isn't a simple "recruit more core developers", as the existing core developers are *also* the bottleneck in the review and mentoring process for *new* core developers.
As a random datapoint: Early last year I wanted get involved in CPython development. In the next months I submitted and reviewed maybe 20 - 25 patches in the bug tracker. After seeing all of them languish, I stopped submitting and reviewing and just tried to get someone to look at the issues that I'd worked on. Eventually, I managed to get almost all of them committed (the last one sometime this February, after more than a year). However, it was such an uphill battle just to get someone to look at my contributions that I haven't touched the bugtracker ever since. If it were up to me, I'd focus all the resources of the PSF on reducing this backlog - be that by hiring some core developers to work full-time on just the open bugtracker issues, or by financing development of better code review and commit infrastructure. The current situation looks like a downward spiral to me. New contributors are frustrated and leave because they feel their contribution is not welcome, and core developers get burned out by the gigantic backlog and the interaction with frustrated patch submitters - thus further reducing the available manpower. (I am aware of the forge PEPs that you mention below, but as far as I know even those are stalled because of - guess what - lack of available core developer time). Working on Roundup, the CPython infrastructure or Kallithea may be less frustrating for new contributors (I don't know), but what this boils down to is that you're contributing to a different project, and not to CPython. But if you've decided to work on something else, there is (at least for me) little reason to restrict the choice to projects that are used *for* CPython development. And compared to the whole range of other open source projects, I suspect the above options just don't look all that interesting to many people. In other words: you typically can't tell volunteers what to work on - you've got to accept the contribution in the area they're interested in, or you loose the contributor. In case of CPython the latter may be unavoidable at the moment, but I think it's illusory to think that this can be solved by "redirecting" the stream of contributions. Suppose you just found a bug in Python and you want to upstream the patch so you don't have to carry the workaround forever. Now you see there are already a lot of open issues with patches - are you going to forget about your patch and e.g. decide to work on the buildbots insteads? Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
On 22 July 2015 at 13:23, Nikolaus Rath <Nikolaus@rath.org> wrote:
If it were up to me, I'd focus all the resources of the PSF on reducing this backlog - be that by hiring some core developers to work full-time on just the open bugtracker issues, or by financing development of better code review and commit infrastructure.
From the "strange but true" files, it's also the case that growing the core developer community *isn't* actually the PSF's highest priority -
Ah, but the PSF can't do that without infringing on python-dev's autonomy - switching to my PSF Director's hat, while we'd certainly be prepared to help with funding a credible grant proposal for something like the Twisted technical fellowship, we wouldn't *impose* help that the core developers haven't asked for. That again bottlenecks on core developer time, though - figuring out how to reasonably run such a fellowship programme (especially the recruiting aspects) without alienating volunteers is yet another "hard community management problem". that honour goes to growing the Python *user* community, and the reference interpreter itself is only one aspect of that (while https://wiki.python.org/moin/PythonSoftwareFoundation/Proposals/StrategicPri... is only a draft, it still gives a good guide as to the extent of the PSF's wider activities).
The current situation looks like a downward spiral to me. New contributors are frustrated and leave because they feel their contribution is not welcome, and core developers get burned out by the gigantic backlog and the interaction with frustrated patch submitters - thus further reducing the available manpower.
We actually still have a lot of paid core developer (and potential core developer) time locked up in facilitating the Python 2 -> 3 migration, as we didn't fully appreciate the extent to which Python had been adopted in the Linux ecosystem and elsewhere until folks started seeking help upgrading. Knowing that doesn't help in the near term, of course, but it does mean we know there's a relatively large pool of paid development time currently being directed elsewhere that may be brought more directly to bear upstream in the future. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jul 22 2015, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 22 July 2015 at 13:23, Nikolaus Rath <Nikolaus@rath.org> wrote:
If it were up to me, I'd focus all the resources of the PSF on reducing this backlog - be that by hiring some core developers to work full-time on just the open bugtracker issues, or by financing development of better code review and commit infrastructure.
Ah, but the PSF can't do that without infringing on python-dev's autonomy - switching to my PSF Director's hat, while we'd certainly be prepared to help with funding a credible grant proposal for something like the Twisted technical fellowship, we wouldn't *impose* help that the core developers haven't asked for.
I don't understand. If I would hire a core developer myself to work on this (theoretically, I have no plans to do that), would that also be infringing python-dev's authority? If so, how is that different from me doing the work? If not, why is it different if the PSF decides to hire someone?
The current situation looks like a downward spiral to me. New contributors are frustrated and leave because they feel their contribution is not welcome, and core developers get burned out by the gigantic backlog and the interaction with frustrated patch submitters - thus further reducing the available manpower.
We actually still have a lot of paid core developer (and potential core developer) time locked up in facilitating the Python 2 -> 3 migration, as we didn't fully appreciate the extent to which Python had been adopted in the Linux ecosystem and elsewhere until folks started seeking help upgrading.
Interesting. Is this information available publically somewhere? I'm curious what exactly is being worked on. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«
On 23 Jul 2015 01:36, "Nikolaus Rath" <Nikolaus@rath.org> wrote:
On Jul 22 2015, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 22 July 2015 at 13:23, Nikolaus Rath <Nikolaus@rath.org> wrote:
If it were up to me, I'd focus all the resources of the PSF on reducing this backlog - be that by hiring some core developers to work full-time on just the open bugtracker issues, or by financing development of better code review and commit infrastructure.
Ah, but the PSF can't do that without infringing on python-dev's autonomy - switching to my PSF Director's hat, while we'd certainly be prepared to help with funding a credible grant proposal for something like the Twisted technical fellowship, we wouldn't *impose* help that the core developers haven't asked for.
I don't understand. If I would hire a core developer myself to work on this (theoretically, I have no plans to do that), would that also be infringing python-dev's authority? If so, how is that different from me doing the work? If not, why is it different if the PSF decides to hire someone?
When somebody else pays someone to work on core development, it's quite clear that that's a private employment matter between that developer and whoever hires them. By contrast, the PSF also has to consider the potential impact on motivation levels for all the current volunteers we *don't* hire, as well as ensuring that expectations are appropriately aligned between everyone involved in the process. I think that's more likely to work out well for all concerned if the process of requesting paid help in keeping the issue tracker backlog under control is initiated *from* the core development community, rather than being externally initiated by the PSF Board.
The current situation looks like a downward spiral to me. New contributors are frustrated and leave because they feel their contribution is not welcome, and core developers get burned out by the gigantic backlog and the interaction with frustrated patch submitters - thus further reducing the available manpower.
We actually still have a lot of paid core developer (and potential core developer) time locked up in facilitating the Python 2 -> 3 migration, as we didn't fully appreciate the extent to which Python had been adopted in the Linux ecosystem and elsewhere until folks started seeking help upgrading.
Interesting. Is this information available publically somewhere? I'm curious what exactly is being worked on.
There are a couple of links for Ubuntu & Fedora porting status at https://wiki.python.org/moin/Python3LinuxDistroPortingStatus Canonical & Red Hat between them have several people working on that, and upgrades for a large proportion of the enterprise Linux world are gated behind that effort. The PyCon US sponsor list then provides a decent hint as to the scale of what's needing to be ported behind corporate firewalls: https://us.pycon.org/2015/sponsors/ It definitely qualifies as interesting times :) Cheers, Nick.
* Nikolaus Rath <Nikolaus@rath.org> [2015-07-21 20:23:15 -0700]:
On Jul 21 2015, Nick Coghlan <ncoghlan@gmail.com> wrote:
All of this is why the chart that I believe should be worrying people is the topmost one on this page: http://bugs.python.org/issue?@template=stats
Both the number of open issues and the number of open issues with patches are steadily trending upwards. That means the bottleneck in the current process *isn't* getting patches written in the first place, it's getting them up to the appropriate standards and applied. Yet the answer to the problem isn't a simple "recruit more core developers", as the existing core developers are *also* the bottleneck in the review and mentoring process for *new* core developers.
As a random datapoint:
Early last year I wanted get involved in CPython development. In the next months I submitted and reviewed maybe 20 - 25 patches in the bug tracker. After seeing all of them languish, I stopped submitting and reviewing and just tried to get someone to look at the issues that I'd worked on. Eventually, I managed to get almost all of them committed (the last one sometime this February, after more than a year). However, it was such an uphill battle just to get someone to look at my contributions that I haven't touched the bugtracker ever since.
[...]
As another random datapoint: I have some (minor) things which I'd like to contribute to Python - and I never did. Seeing the number of open issues with patches just make me feel like it'd be a waste of time to contribute. It seems very plausible the patches will just be ignored without me putting effort in getting them noticed. I'm fine with revising things until people are happy, i.e. I don't just want to post a patch and disappear - but I don't want to fight to get any kind of response, and it looks to me like I'd have to. :-/ Florian -- http://www.the-compiler.org | me@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/
participants (11)
-
Alexander Belopolsky -
Baptiste Carvello -
Ben Finney -
Ethan Furman -
Florian Bruhin -
Nick Coghlan -
Nikolaus Rath -
Paul Moore -
Robert Collins -
Stephen J. Turnbull -
Wes Turner