Learning from PostgreSQL community: How to address the review bottleneck
I've attended FOSDEM over the weekend, where Jon Conway (one of the PostgreSQL committers) gave a talk about, among other things, the PG community and how it is structured:
https://fosdem.org/2019/schedule/event/postgresql11/ (the community part starts at around 8 min into the video)
What struck me as interesting is that they have seen and addressed the review bottleneck problem we're having in Python development years ago.
They have a core team, which pretty much resembles the steering committee we've just voted on, with 5 members, and a group of 28 committers. Things are much less formalized than in Python land, but they are making great progress.
Here's their approach to solve the review bottleneck:
https://wiki.postgresql.org/wiki/PostgreSQL_8.4_Development_Plan (they started this in 2008)
with what they call "commit fests". This is a process where people submit patches using timed slots, each author is requested to do at least one review of another patch of similar complexity and the authors can fix their patches as part of the review process to get them to a level where a core dev can than take a look. Other people can sign up as reviewers as well.
That way the initial load of making sure the patch quality is appropriate is scaled up a lot and their core devs only have to deal with patches which already have passed reviews by a few people.
The process is described in more detail in this blog post:
https://blog.2ndquadrant.com/managing-a-postgresql-commitfest/ (with the experience after doing this for 8 years)
To help them with the commit fests, they have a system in place to manage the patches:
https://commitfest.postgresql.org/
See e.g. https://commitfest.postgresql.org/22/ for the next upcoming commit fest.
Commit fests are done for one month each, and then leave one month for things to settle in, get core dev responses. Patches can be pushed back to the next commit fest in case a core dev finds them lacking or the author doesn't respond in time.
I also talked to Magnus Hagander, one of the PG core team members, about their core team. They have had this since the early 2000s and interestingly, they are mostly dealing with non-developer questions. Their approach to decisions such as the PEP process we have is mostly based on consensus and trust among the committers, not formalized and thus the core team does not play into this a lot.
https://www.postgresql.eu/events/pgdayparis2018/schedule/speaker/1-magnus-ha...
Now, all that said, while there are many similarities between PostgreSQL and Python in how the communities work, PG does take a more conservative approach to things - most committers and core team members have had that status for at least 10 years, it typically takes several years to gain committer status and they rarely take on new people.
Still, I think there's a lot we can learn from them and their experience with solving the review problem.
Thanks,
Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Experts (#1, Feb 04 2019)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Hi Marc-André,
I find this feedback very interesting :-)
As PG is a sophisticated piece of high-quality software, if that process works for them, then it may deserve trying on our side as well.
Regards
Antoine.
Le 04/02/2019 à 12:03, M.-A. Lemburg a écrit :
I've attended FOSDEM over the weekend, where Jon Conway (one of the PostgreSQL committers) gave a talk about, among other things, the PG community and how it is structured:
https://fosdem.org/2019/schedule/event/postgresql11/ (the community part starts at around 8 min into the video)
What struck me as interesting is that they have seen and addressed the review bottleneck problem we're having in Python development years ago.
They have a core team, which pretty much resembles the steering committee we've just voted on, with 5 members, and a group of 28 committers. Things are much less formalized than in Python land, but they are making great progress.
Here's their approach to solve the review bottleneck:
https://wiki.postgresql.org/wiki/PostgreSQL_8.4_Development_Plan (they started this in 2008)
with what they call "commit fests". This is a process where people submit patches using timed slots, each author is requested to do at least one review of another patch of similar complexity and the authors can fix their patches as part of the review process to get them to a level where a core dev can than take a look. Other people can sign up as reviewers as well.
That way the initial load of making sure the patch quality is appropriate is scaled up a lot and their core devs only have to deal with patches which already have passed reviews by a few people.
The process is described in more detail in this blog post:
https://blog.2ndquadrant.com/managing-a-postgresql-commitfest/ (with the experience after doing this for 8 years)
To help them with the commit fests, they have a system in place to manage the patches:
https://commitfest.postgresql.org/
See e.g. https://commitfest.postgresql.org/22/ for the next upcoming commit fest.
Commit fests are done for one month each, and then leave one month for things to settle in, get core dev responses. Patches can be pushed back to the next commit fest in case a core dev finds them lacking or the author doesn't respond in time.
I also talked to Magnus Hagander, one of the PG core team members, about their core team. They have had this since the early 2000s and interestingly, they are mostly dealing with non-developer questions. Their approach to decisions such as the PEP process we have is mostly based on consensus and trust among the committers, not formalized and thus the core team does not play into this a lot.
https://www.postgresql.eu/events/pgdayparis2018/schedule/speaker/1-magnus-ha...
Now, all that said, while there are many similarities between PostgreSQL and Python in how the communities work, PG does take a more conservative approach to things - most committers and core team members have had that status for at least 10 years, it typically takes several years to gain committer status and they rarely take on new people.
Still, I think there's a lot we can learn from them and their experience with solving the review problem.
Thanks,
participants (2)
-
Antoine Pitrou -
M.-A. Lemburg