As specifically to the flaws in our workflow and the backlog, this is exactly what the Developer-in-Residence program was designed to address: https://pyfound.blogspot.com/2021/04/the-psf-is-hiring-developer-in.html Stay tuned! -Barry

On Jun 29, 2021, at 09:56, Joannah Nanjekye <nanjekyejoannah@gmail.com> wrote:

...

why doesn't it get merged?

The last significant discussion from a core dev on the most relevant PR here: https://github.com/python/cpython/pull/4819 shows that Antoine was familiarizing himself with the feature and had added it to their to do list.

Merging something is also a responsibility to whoever does it, so I suggest we approach this with grace given at least someone promised to look into it.

BTW, am not writing off your concerns.

On Tue, Jun 29, 2021 at 1:36 PM <esmeraldagarcia@byom.de> wrote: I just stumbled upon the following issue and subsequent pull request. It is a very small bugfix. There is currently a bug in Python and this pull request fixes it. It's not a new feature or an enhancement, it is a bugfix! Yet, it doesn't get reviewed, nor merged. And this has been going on since March 2017. Why not just merge this? It's not like it's huge or obstructing or obfuscating the current code base? There's always time to write an improvement or a complete rewrite of this module feature later for an upcoming minor release. But if there is currently a bug in Python and the bugfix is available - why doesn't it get merged?

https://github.com/python/cpython/pull/4819

If this doesn't get fixed, doesn't that mean the Python review process is flawed? Sure, Python is an open source project and many people just work in their free time. But this doesn't really apply here, does it? The bugfix is available. Only a review is required. All this is happening while new features get added to Python with every new minor version. While the bug is allowed to live there. Please help me understand how this can happen.

I love Python. No hard feelings. But this is really bugging me and I can't help but feel disappointed. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TTJAVTPF... Code of Conduct: http://python.org/psf/codeofconduct/

-- Best, Joannah Nanjekye "You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program." Alan J. Perlis _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BCGNYV73... Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Jul 1, 2021 at 10:49 AM Victor Stinner <vstinner@python.org> wrote:

What happens usually is that some modules have no maintainer. Once you merge a single change in a module, boom! You are now the new maintainer for your entire life time! More and more people will ask you to look at their "very important" bug blocking their production and their very specific use case. You will get more and more reviews request: "since you merged a single change 12 months ago, I'm sure that you will love to review my precioooous bugfix! Come on, it's trivial!". Now you realize that you don't know the design of the module. You don't know its history. You know nothing, and the entire world now have very high expectation, since you merged an obvious and trivial change.

“The ancient Oracle said that I was the wisest of all the Greeks. It is because I alone, of all the Greeks, know that I know nothing.” ~ Socrates The same could largely be applied to maintaining open source (especially a codebase as large as CPython). The truth is that at a fundamental level, we're all (any software developer) just making educated guesses as to whether or not patches are suitable for merging, or whether or not a bug fix will actually work. Tests make us a little more certain, but it is still a guess. So the problem is really just the unrealistic expectation that there exists people who are able to make a decision with absolute 100% certainty that no regressions or future unexpected issues will occur as a result of the change. Also, for fun, here's a pythonized version of the quote: “[Guido] said that I was the wisest of all the [pythonistas]. It is because I alone, of all the [pythonistas], know that I know nothing.” :) With loving-kindness, -- --Kyle R. Stanley, Python Core Developer (what is a core dev? <https://devguide.python.org/coredev/>) *Pronouns: they/them **(why is my pronoun here?* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...> )

Okay so I just code a little bit and I used the multiprocessing module but my code didn't work and I found the solution on Stack Overflow and it turned out to be not my mistake (which has never happened before I think). Instead I found out it's a bug in Python and the issue on Github was linked so I opened it and I was surprised to see what's going on "behind the scenes". Yes I have basically no experience in maintaining any big project. So when you're saying "You don't know what it's like and therefore your complaint doesn't make sense" then you're not wrong and I just have to believe you. But I think this is a dangerous argument because it could also be used to shut up anything and anybody. (I'm not saying this is the case here.) Therefore, this argument should rarely be used in my opinion. From an outsider's perspective it just looks really weird that a bugfix from 2017 hasn't become a priority to get merged, like the process is flawed. That's all. I didn't mean to attack any one of you. I want to make that clear because it feels like some of you got kinda defensive about it. "There's been quite a bit of discussion on both of them" - None of the discussions left any questions unanswered. Except for the question of when the pull request will get merged. "Merging something is also a responsibility to whoever does it" - And it's also a responsibility to fix bugs, no? I don't get why you're so afraid of (maybe!) introducing a new bug when there already (certainly!) is a bug. "Oops. I'm really sorry for giving false hopes, then, because I don't think I'm motivated to review this PR. I'm not really maintaining multiprocessing these days, anymore" - No worries dude. This not about one person or one bug. I'm sorry that the issue that I stumbled upon turned out to be one where you said you'd put it on your list. "What if that one line change is even more wrong than before?" - Yes of course there's a risk. Just like there was a risk when you merged the original code which contained the bug, right?! At some point you have to say yes that looks okay let's merge it, even though there is a slight chance it could contain a mistake. And it is not obvious to me (and many other people who commented in those github threads) what else would possibly be needed. After all, there are currently actual people who are affected by the bug - and you're only talking about hypothetical people being affected by a possibly wrong bugfix. "When I got the shutil.which feature merged, the PR had been open for I believe 11 years" - Totally different topic. I explicitly said in my initial message, that I'm talking about a bugfix, not a new feature. "If you would like more value out of it or to speed up the process, you can provide your own reviews." - Seriously? I can't help but feel like that comment sounds kinda arrogant. I hope I'm misunderstanding you. Look at that link and Stack Overflow post again how many people commented and voted that the patch fixed their issues. How many more people do you want? "*maintainer attention* is actually the scarcest resource in many open source projects, and Python is no exception." - Then get more people to do this? Don't tell me Python isn't big enough to find some companies or funds to sponsor a few people to work the dreaded reviewer job a few hours a week? Or let more amateur coders review and have a core reviewer review their reviews? I totally get the point that reviewers are a scarce resource. But I do not get the point why you're not changing that. "almost by definition, ANY regression is worse than any bug that still exists today" - Yeah I get this point and I think I agree. But it's more about risk evaluation. Because if there is absolutely no willingness to risk mistakenly introducing a regression then you're effectively at a standstill. You can never merge anything again because it might affect the code base in a way you hadn't foreseen. So you need to take some risks. "Most stdlib modules have no maintainer and past maintainers are gone for a long time." - I'm flabbergasted. I don't know what to say. Can't you not see how bad that is?! "As specifically to the flaws in our workflow and the backlog, this is exactly what the was designed to address" - To end on a positive note: The Developer-in-Residence program sounds like a really good idea. And I love Python and appreciate all the work that went into it and I really hope all of you believe me when I say this despite me internet rando pooping on your review process. <3

On Fri, Jul 2, 2021 at 10:06 AM Eric V. Smith <eric@trueblade.com> wrote:

On 7/1/2021 7:39 PM, esmeraldagarcia@byom.de wrote:

...

"Merging something is also a responsibility to whoever does it" - And it's also a responsibility to fix bugs, no? I don't get why you're so afraid of (maybe!) introducing a new bug when there already (certainly!) is a bug.

Because the new bug you introduce might be much, much worse than the bug you're fixing.

It's worth noting that, in the (rare) cases where there clearly is no downside to fixing a bug... it just gets fixed. You won't see those kinds of issues in Stack Overflow answers, because they're not there any more. Calling the review process flawed on the basis of the issues still open after X years is as incomplete a picture as saying that all music from the 18th century is awesome on the basis of the music that's still being played today. *By definition*, you're only seeing the hairy issues, the ones where it's far from clear what the correct action is (merge, modify, or do nothing). If you want a rough idea of the things that get fixed quietly, browse the What's New for a point release, for example: https://www.python.org/downloads/release/python-394/ ChrisA

On Thu, Jul 1, 2021 at 4:54 PM <esmeraldagarcia@byom.de> wrote:

Okay so I just code a little bit and I used the multiprocessing module but my code didn't work and I found the solution on Stack Overflow and it turned out to be not my mistake (which has never happened before I think). Instead I found out it's a bug in Python and the issue on Github was linked so I opened it and I was surprised to see what's going on "behind the scenes".

Yes I have basically no experience in maintaining any big project. So when you're saying "You don't know what it's like and therefore your complaint doesn't make sense" then you're not wrong and I just have to believe you. But I think this is a dangerous argument because it could also be used to shut up anything and anybody. (I'm not saying this is the case here.) Therefore, this argument should rarely be used in my opinion. From an outsider's perspective it just looks really weird that a bugfix from 2017 hasn't become a priority to get merged, like the process is flawed. That's all. I didn't mean to attack any one of you. I want to make that clear because it feels like some of you got kinda defensive about it.

We do get defensive because people on a regular basis come to us asking, "why haven't you fixed this?" and we have to explain why, and some people accept the answer and others continue to push us which in all honesty becomes exhausting. Add on that not everyone asking in an understanding, kind way and it makes one not want to even reply most of the time.

"There's been quite a bit of discussion on both of them" - None of the discussions left any questions unanswered. Except for the question of when the pull request will get merged.

"Merging something is also a responsibility to whoever does it" - And it's also a responsibility to fix bugs, no? I don't get why you're so afraid of (maybe!) introducing a new bug when there already (certainly!) is a bug.

"Oops. I'm really sorry for giving false hopes, then, because I don't think I'm motivated to review this PR. I'm not really maintaining multiprocessing these days, anymore" - No worries dude. This not about one person or one bug. I'm sorry that the issue that I stumbled upon turned out to be one where you said you'd put it on your list.

"What if that one line change is even more wrong than before?" - Yes of course there's a risk. Just like there was a risk when you merged the original code which contained the bug, right?! At some point you have to say yes that looks okay let's merge it, even though there is a slight chance it could contain a mistake. And it is not obvious to me (and many other people who commented in those github threads) what else would possibly be needed. After all, there are currently actual people who are affected by the bug - and you're only talking about hypothetical people being affected by a possibly wrong bugfix.

As Eric points out in his own follow-up, sometimes the fix is worse than the bug it's fixing. Add to that the feeling when people descend upon you for breaking their code that was working due to a "bugix" which from their view wasn't, and it makes you very cautious about accepting any PR w/o having a solid understanding of the ramifications. I have people come up to me at PyCon US about code I have changed over a decade ago to complain about it. It's exhausting sometimes.

"When I got the shutil.which feature merged, the PR had been open for I believe 11 years" - Totally different topic. I explicitly said in my initial message, that I'm talking about a bugfix, not a new feature.

"If you would like more value out of it or to speed up the process, you can provide your own reviews." - Seriously? I can't help but feel like that comment sounds kinda arrogant. I hope I'm misunderstanding you. Look at that link and Stack Overflow post again how many people commented and voted that the patch fixed their issues. How many more people do you want?

A SO comment is not a code review. What was being requested is someone not only validating the fix but making sure the code is solid, meets coding guidelines, has appropriate tests, etc. There is much more to maintaining the code than just whether it functions appropriately.

"*maintainer attention* is actually the scarcest resource in many open source projects, and Python is no exception." - Then get more people to do this? Don't tell me Python isn't big enough to find some companies or funds to sponsor a few people to work the dreaded reviewer job a few hours a week?

Welcome to open source and why it is perpetually underfunded. The PSF just got funding for the first time in Python's 30 year history this year to hire someone to work on Python full-time. I get 20% for open source from my employer and I am very lucky to get that plus whatever time I spend away from family and friends in my spare time (like now while I reply to this during a holiday). Or put another way: does your employer pay for you or anyone else to contribute to Python? Or are they a sponsor of the PSF?

Or let more amateur coders review and have a core reviewer review their reviews?

Anyone can do code reviews; it's one of the reasons we are on GitHub and someone asked you to please provide a code review.

I totally get the point that reviewers are a scarce resource. But I do not get the point why you're not changing that.

We can't force people to give us funding, or force companies to hire folks to work on Python, or make people give code reviews. I have personally put it in a vast amount of my personal time trying to lower the barrier of entry for people on this project as well as make reviews easier. But you simply can't will people to help out.

"almost by definition, ANY regression is worse than any bug that still exists today" - Yeah I get this point and I think I agree. But it's more about risk evaluation. Because if there is absolutely no willingness to risk mistakenly introducing a regression then you're effectively at a standstill. You can never merge anything again because it might affect the code base in a way you hadn't foreseen. So you need to take some risks.

Yes, but you have to balance that risk out, and that risk calculation may or may not go the way you want (as you're discovering).

"Most stdlib modules have no maintainer and past maintainers are gone for a long time." - I'm flabbergasted. I don't know what to say. Can't you not see how bad that is?!

Sure, which is why we cleaned up the stdlib when we transitioned to Python 3.0 and we have https://www.python.org/dev/peps/pep-0594/ under consideration. But do understand that if we dropped every module that lacked a direct maintainer there wouldn't be much of a stdlib anymore (which perhaps you're fine with; everyone has an opinion on this topic and I will be starting a chat on this in the future 😅).

"As specifically to the flaws in our workflow and the backlog, this is exactly what the was designed to address" - To end on a positive note: The Developer-in-Residence program sounds like a really good idea. And I love Python and appreciate all the work that went into it and I really hope all of you believe me when I say this despite me internet rando pooping on your review process. <3

I think a key thing to keep in mind is this has been our overall dev process of literally decades and it has produced this software that you love. So do understand that while you may be shocked by "how the sausage is made", it has gotten you something you have admitted you love overall. If you want to read more about the difficulties in balancing all of this, I've written and spoken about it extensively, e.g. https://snarky.ca/setting-expectations-for-open-source-participation/ and https://snarky.ca/the-social-contract-of-open-source/.

Okay so I just code a little bit and I used the multiprocessing module but my code didn't work and I found the solution on Stack Overflow and it turned out to be not my mistake (which has never happened before I think). Instead I found out it's a bug in Python and the issue on Github was linked so I opened it and I was surprised to see what's going on "behind the scenes".

Yes I have basically no experience in maintaining any big project. So when you're saying "You don't know what it's like and therefore your complaint doesn't make sense" then you're not wrong and I just have to believe you. But I think this is a dangerous argument because it could also be used to shut up anything and anybody. (I'm not saying this is the case here.) Therefore, this argument should rarely be used in my opinion. From an outsider's perspective it just looks really weird that a bugfix from 2017 hasn't become a priority to get merged, like the process is flawed. That's all. I didn't mean to attack any one of you. I want to make that clear because it feels like some of you got kinda defensive about it.

I don't think anyone felt attacked. The hard-working devs are merely

discussions left any questions unanswered. Except for the question of when the pull request will get merged.

"Merging something is also a responsibility to whoever does it" - And it's also a responsibility to fix bugs, no? I don't get why you're so afraid of (maybe!) introducing a new bug when there already (certainly!) is a bug.

Whose "responsibility" do you think it should be to fix bugs? Who do you

On Fri, Jul 2, 2021 at 12:47 AM <esmeraldagarcia@byom.de> wrote: trying to explain the facts of life, and if exasperation occasionally creeps in that's probably because this is far from the first time this issue has been raised here. (You'll note I refer to the core devs in the third person, since I am not one of them and neither do I speak for them, I am merely recording my observations). "There's been quite a bit of discussion on both of them" - None of the think should set priorities to determine which work is done first. Few core developers are paid to work on Python, and even those that are (until the PSF's developer is appointed) might expect to have their overall priorities set by their employer. As a consumer of their work who's made little contribution to the language I don't personally feel that I have much right to dictate how devs spend their time. I'm just glad so many of them do. The fact is that for any community-run open source project the only reliable way to ensure PRs get merged is to acquire commit rights and do it yourself. It's by no means ideal, but that's the current reality for Python. "Oops. I'm really sorry for giving false hopes, then, because I don't think

I'm motivated to review this PR. I'm not really maintaining multiprocessing these days, anymore" - No worries dude. This not about one person or one bug. I'm sorry that the issue that I stumbled upon turned out to be one where you said you'd put it on your list.

"What if that one line change is even more wrong than before?" - Yes of course there's a risk. Just like there was a risk when you merged the original code which contained the bug, right?! At some point you have to say yes that looks okay let's merge it, even though there is a slight chance it could contain a mistake. And it is not obvious to me (and many other people who commented in those github threads) what else would possibly be needed. After all, there are currently actual people who are affected by the bug - and you're only talking about hypothetical people being affected by a possibly wrong bugfix.

Let's assume that it takes an hour to properly review and merge a PR. If someone only has five hours a week to work on Python they are hardly going to consider spending 20% of their available time tht week on sometthing unrelted to the work they've been doint of rk

"When I got the shutil.which feature merged, the PR had been open for I believe 11 years" - Totally different topic. I explicitly said in my initial message, that I'm talking about a bugfix, not a new feature.

"If you would like more value out of it or to speed up the process, you can provide your own reviews." - Seriously? I can't help but feel like that comment sounds kinda arrogant. I hope I'm misunderstanding you. Look at that link and Stack Overflow post again how many people commented and voted that the patch fixed their issues. How many more people do you want?

It isn't a matter of summoning the desire, it's a matter of allocating time.

"*maintainer attention* is actually the scarcest resource in many open source projects, and Python is no exception." - Then get more people to do this? Don't tell me Python isn't big enough to find some companies or funds to sponsor a few people to work the dreaded reviewer job a few hours a week? Or let more amateur coders review and have a core reviewer review their reviews? I totally get the point that reviewers are a scarce resource. But I do not get the point why you're not changing that.

You appear to assume that to do so is a simple matter. As someone who's literally been responsible for the running of the Python Software Foundation I can tell you it isn't. Where would these amateur coders you describe come from? How much would that shorten the process if their reviews still need review? How will the reviewers be recruited and trained?

exists today" - Yeah I get this point and I think I agree. But it's more about risk evaluation. Because if there is absolutely no willingness to risk mistakenly introducing a regression then you're effectively at a standstill. You can never merge anything again because it might affect the code base in a way you hadn't foreseen. So you need to take some risks.

"Most stdlib modules have no maintainer and past maintainers are gone for a long time." - I'm flabbergasted. I don't know what to say. Can't you not see how bad that is?!

I don 't think anyone is in much doubt about that. But you still seem to

"almost by definition, ANY regression is worse than any bug that still think that resources can be magically conjured up somehow. They can't, and it's a little offensive to assume that your perceptions of the project will somehow be revelatory. You describe real problems, but these problems are well-known to the development team and you offer no new insights or solutions. It's great you see the same problems the core devs do, but understanding the problem alone isn't going to achieve anything without additional resources. "As specifically to the flaws in our workflow and the backlog, this is

exactly what the was designed to address" - To end on a positive note: The Developer-in-Residence program sounds like a really good idea. And I love Python and appreciate all the work that went into it and I really hope all of you believe me when I say this despite me internet rando pooping on your review process. <3

That's perfectly believable. Kind regards, Steve