Hi Tal, Thanks for driving such research, it's really interesting. Sadly, I'm not surprised by the results, since I'm already aware of these issues. IMHO the most promising way to get more review is to create groups where developers work on the same topic and review each other. Also, mentoring is great! I advice core developers to delegate their "boring tasks" to people willing to help. Many contributors are eager to take these new responsibilities. Delegating shortly gives more time to work on other topics where rare skills are needed.

* Hard to find where to contribute

I believe that https://devguide.python.org/ could be enhanced to better explain this part. Here are my notes on this question: "Contribute to CPython: where should I start?" https://cpython-core-tutorial.readthedocs.io/en/latest/where_should_i_start....

* Mentorship helps a lot, but is scarce

Over the last 2 years, I mentored different contributors, some of them have been promoted as core developers, some of them moved to another project, some of them continue to contribute as contributors. Mentoring really accelerates the training to learn the most critical things that a developer must know about the Python development workflow, but also to build a trust relationship. I'm trying to communicate about mentorship and my success stories, but I chose to not communicate about people that I'm mentoring since it would put too much pressure on them. It's perfectly fine if a mentee doesn't become a core dev. The difficult part is to stop doing simple tasks and delegate them. It's difficult since mentoring does take more time in the short term, reviews require a lot of back-and-forth exchanges to reach the expected code quality. It's hard to foresee the benefit in the long term. In the past, I complained a lot about failing buildbots, the maintenance burden of the servers and analyzing every single failure. Thanks to mentoring, Pablo decided to help and I'm now really happy to distribute the workload with him! For one year, I'm working on https://bugs.python.org/issue1635741 which is a giant project (issue open for 13 years, tons of sub-issues, tons of commits). The idea is to modernize the code base of the 118 C extension modules of the Python standard library: convert static types to heap types, and convert extensions to the multi-phase initialization API (PEP 489). Required changes are complex and must be reviewed carefully. Also, a part of this work is boring since it's always the same things (repeat the same patterns). Dong-hee Na, Hai Shi, Mohamed Koubaa are collaborating with me, and we made significant progress in one year. When one of them writes a PR, I ask the others to review it. Since we are a small task force working on the same topic for one year, slowly everyone is able to spot the common mistakes. For example, forget to DECREF the type in the dealloc function when a static type becomes a heap type. I trust them and so it puts less pressure on my shoulders when I have to merge a PR. I also spend less time on reviewing, and no time on writing changes since others are writing these changes! IMO this collaboration is a success since the scope is limited, it's possible to work on a single file per PR (or even write multiple PRs per file). While changes are complex, once you understand what should be done, it's mostly a simple task of replacing a pattern with another (ok, it's a little bit more complex than that in practice ;-)). Le sam. 17 oct. 2020 à 00:30, Tal Einat <taleinat@gmail.com> a écrit :

1. Generate reminders for reviewers (...)

The root issue is the number of reviewers. Right now, it's mostly core devs who review changes. I'm not sure that forcing core developers will solve the root issue.

3. Advertise something like a "2-for-1" standing offer for reviews.

Right. That sounds like a better long term solution. We need to encourage contributors to review pull requests written by other contributors in similar areas. As I wrote, when I read a review from a contributor, even if they are a complete stranger to me, I'm more confident in reviewing a PR, since I expect that the first review layers have been checked.

4. Encourage core devs to dedicate some of their time to working through issues/PRs which are "ignored" or "stalled".

I ignore many areas of Python on purpose. I don't have skills in these areas or simply no interest. I have a different idea. What about organizing weeks of review where we focus on a small set of topics. For example, organize an "IDLE" week where we invite everybody to review IDLE issues, triage IDLE bugs, or maybe write new PRs. To be efficient, we need core developers involved in the loop to merge PRs, core devs expert in this topic. Otherwise, we cannot reduce the number of open PRs. I propose a whole week since reviews always need many back-and-forth. Such week might be the opportunity for more information discussions on a dedicated chat where people can ask any question, especially dumb questions. I'm a volunteer to help for such a week, but don't expect me to be available more than 1h per day, and only from monday to friday (so 5 days, 1h per day). Another idea would be to open a video call where I share my screen to share what I'm doing, but I'm not sure if it's interesting :-) Example of topics: * Review bpo-1635741 PRs and write new PRs (need C skills) * Investigate test_asyncio random failures on buildbots Sometimes, such collaboration doesn't produce any "value" immediately, but it always helps people to meet each other and feel less shy to ask questions and reach out to other people. What do you think? Victor -- Night gathers, and now my watch begins. It shall not end until my death.

On 10/16/20 3:29 PM, Tal Einat wrote:

(Context: Continuing to prepare for the core dev sprint next week. Since the sprint is near, *I'd greatly appreciate any quick comments, feedback and ideas!*)

Following up my collection of past beginning contributor experiences, I've collected these experiences in a dedicated GitHub repo[1] and written a (subjective!) summary of main themes that I recognize in the stories, which I've also included in the repo[2].

A "TL;DR" bullet list of those main themes: * Slow/no responsiveness * Long, slow process * Hard to find where to contribute * Mentorship helps a lot, but is scarce * A lot to learn to get started * It's intimidating

More specifically, something that has come up often is that maintaining momentum for new contributors is crucial for them to become long-term contributors. Most often, this comes up in relation to the first two points: Suggestions or PRs are completely receive no attention at all ("ignored") or stop receiving attention at some point ("lost to the void"). Unfortunately, the probability of this is pretty high for any issue/PR, so for a new contributor this is almost guaranteed to happen while working on one of their first few contributions. I've seen this happen many times, and have found that I have to personally follow promising contributors' work to ensure that this doesn't happen to them. I've also seen contributors learn to actively seek out core devs when these situations arise, which is often a successful tactic, but shouldn't be necessary so often.

Now, this is in large part a result of the fact that us core devs are not a very large group, made up almost entirely of volunteers working on this in their spare time. Last I checked, the total amount of paid development time dedicated to developing Python is less than 3 full-time (i.e. ~100 hours a week).

The situation being problematic is clear enough that the PSF had concrete plans to hire paid developers to review issues and PRs. However, those plans have been put on hold indefinitely, since the PSF's funding has shrunk dramatically since the COVID-19 outbreak (no PyCon!).

So, what can be done? Besides raising more funds (see a note on this below), I think we can find ways to reduce how often issues/PRs become "stalled". Here are some ideas:

1. *Generate reminders for reviewers when an issue or PR becomes "stalled' due to them.* Personally, I've found that both b.p.o. and GitHub make it relatively hard to remember to follow up on all of the many issues/PRs you've taken part in reviewing. It takes considerable attention and discipline to do so consistently, and reminders like these would have helped me. Many (many!) times, all it took to get an issue/PR moving forward (or closed) was a simple "ping?" comment.

2. *Generate reminders for contributors when an issue or PR becomes "stalled" due to them.* Similar to the above, but I consider these separate.

3. *Advertise something like a "2-for-1" standing offer for reviews.* This would give contributors an "official", acceptable way to get attention for their issue/PR, other than "begging" for attention on a mailing list. There are good ways for new contributors to be of significant help despite being new to the project, such as checking whether old bugs are still relevant, searching for duplicate issues, or applying old patches to the current code and creating a PR. (This would be similar to Martin v. Löwis's 5-for-1 offer in 2012[3], which had little success but lead to some interesting followup discussion[4]).

4. *Encourage core devs to dedicate some of their time to working through issues/PRs which are "ignored" or "stalled".* This would require first generating reliable lists of issues/PRs in such states. This could be in various forms, such as predefined GitHub/b.p.o. queries, a dedicated web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over "ignored".)

- Tal Einat

Tal, this would be great. Is it possible to get a ballpark idea on how long a PR should/would take to be merged so that contributors know if they should ping or not? -Alfred