Proposal: "Lightweight" governance model for PyPA

Hello PyPA Committers! I'd like to start discussion on a possible governance model for the PyPA. ## Summary The model described here is what I consider a minimum viable governance for the Python Packaging Authority that would open realtime and asynchronous channels of communication for two purposes: - Keeping PyPA projects up to date on what one another are up to, and where they are headed. - Allowing the PSF Packaging Working Group insight into the PyPA's activities. This model is intended for coordination and communication among Python Packaging Authority projects and does not have bearing on the existing project structure or governance of individual projects beyond the existing requirement of adhering to the PyPA Code of Conduct. ## Model Recurring (weekly, monthly, or quarterly) synchronous meeting of the PyPA via a communications channel TBD. This channel should allow for a log of discussions and some moderation. Structure of meeting TBD, but a something along the lines of: Announcements, Old Business, New Business. A moderator and minute-taker would be responsible for summarizing the meeting, collecting project updates, and publishing minutes and updates for archival purposes and public review. Meeting note publication method TBD. Adherence to the PyPA Code of Conduct would be required for all attendees. ### Requirements #### For attendance as an observer: None (Note that observers can read/watch/listen the meeting in realtime but cannot participate/speak in the meeting.) #### For participation: - Maintainer of a PyPA project. - Contributed a summary of activity for the project ahead of meeting. - Contributed agenda items for Announcements or New Business. OR - Member of Python Software Foundation Packaging Working Group. ## Goals The basic design parameters I had in mind were: - Maintain independence of individual PyPA projects. - Provide a cadence to the PyPA's communications. - Create an incentive for centralized reporting of PyPA project activities. - Provide Packaging Working Group a way to understand what is going on in various PyPA projects, to understand how and where grants could make an impact.

On Tue, 5 Jun 2018, 07:03 Ernest W. Durbin III, <ewdurbin@gmail.com> wrote:
Monthly meetings (which can be skipped if there's nothing to discuss) sound good to me. Structure of meeting TBD, but a something along the lines of:
Announcements, Old Business, New Business.
LGTM as is. :) A moderator and minute-taker would be responsible for summarizing the
meeting, collecting project updates, and publishing minutes and updates for archival purposes and public review. Meeting note publication method TBD.
Question: Process to choose these people? Is only one person incharge of both duties? Adherence to the PyPA Code of Conduct would be required for all attendees.
#### For participation:
I initially had some reservations about restricting participation of Maintainers but giving it a bit more thought, these conditions do look reasonable to me. OR
I like this model. I think the TBDs would be up for discussion, if we do decide to go through with this, so, I refrained from providing suggestions for them now.

Comments on the proposal itself below, but I wanted to sort of diverge a bit here first. I think our first order of business should basically be to figure out some sort of long term decision making process. In my eyes, things like some sort of communication (realtime, async, whatever) or any of our other really open questions sort of should generally a back seat to the question of how do we, as a group, make decisions. We currently have the “BDFRN” distinction, but I view that as a fairly adhoc idea, sort of like “emergency powers” where the ultimate goal should be to to define our real decision making process, and then transition us to that. Once we have that, then I think we should start focusing on communication and other structure.
This sounds reasonably OK to me as long as we tune the recurrence correctly. My primary concern is that the PyPA is getting larger (and is likely to continue to do so), and we have members from all across the globe, so scheduling a single time when everyone who wants to participate is able to could prove difficult. To my knowledge we basically have people on complete opposite ends of the world, and everywhere in between now. We likely also want to add in some mechanism to prevent these from becoming very long or trying to boil the ocean (particularly the first couple are likely to be more prone to this).

On 6 June 2018 at 15:02, Donald Stufft <donald@stufft.io> wrote:
Comments on the proposal itself below, but I wanted to sort of diverge a bit here first.
I think our first order of business should basically be to figure out some sort of long term decision making process. In my eyes, things like some sort of communication (realtime, async, whatever) or any of our other really open questions sort of should generally a back seat to the question of how do we, as a group, make decisions. We currently have the “BDFRN” distinction, but I view that as a fairly adhoc idea, sort of like “emergency powers” where the ultimate goal should be to to define our real decision making process, and then transition us to that. Once we have that, then I think we should start focusing on communication and other structure.
Agreed on this.
I'm personally not keen on the key process being a real-time one. My personal open source time is structured in such a way that I'd basically never be able to attend something like this. I assume/hope that there would be some sort of mechanism for whatever was discussed to be available to everyone (ideally by being mailed out - I prefer push-style communications over pull-style "the logs are here, go & look") with a means for people who didn't attend (but are valid "participants" as per the description) to comment/contribute after the fact. But as currently stated, there doesn't seem to be anything like that, which bothers me. My personal preference would be that we stick to an asynchronous means of discussion/communication, like email. I've seen no evidence that real-time discussions as a *general* mechanism[1] produce better results, and I have definitely seen evidence that they leave people unable to attend feeling at least somewhat disenfranchised. But I'm happy to go with the majority, as long as attendance is not necessary in order to be involved and listened to on an equal footing with people who *do* attend. Paul [1] Having a chat-style option for cases where real-time communication is useful would be fine. But I'm not sure I see evidence that the PyPA is big enough or has sufficiently urgent issues to warrant something like that.

The real time meet ups main advantage is getting decisions made faster and having the ability to be more direct answering peoples concerns, and, thus allowing us to be more efficient. I do share Pauls concerns about people being able to attend, with timezones and this organization being primarily volunteer based. I like Ernest's idea of a sync tho. There could possibly be notes posted from those who can't attend prior and the meeting char (we could rotate) could represent and get the discussion started. We could all take turns at being the scribe and push out notes - like Sumana did for the Warehouse syncs. Then we always have a follow up async communication (e.g. like this). I've also seen that when working with a global team, offering different time zone friendly times will allow different people attend each month. Just some thoughts. I like the discussion. Cooper

On 6 June 2018 at 19:07, Cooper Ry Lees <me@cooperlees.com> wrote:
The real time meet ups main advantage is getting decisions made faster and having the ability to be more direct answering peoples concerns, and, thus allowing us to be more efficient. I do share Pauls concerns about people being able to attend, with timezones and this organization being primarily volunteer based.
Faster decisions pretty much conflicts directly with involving everyone :-( Do we actually *have* decisions that we need to make quickly? I'm not sure on pip we ever have...
I like Ernest's idea of a sync tho. There could possibly be notes posted from those who can't attend prior and the meeting char (we could rotate) could represent and get the discussion started. We could all take turns at being the scribe and push out notes - like Sumana did for the Warehouse syncs. Then we always have a follow up async communication (e.g. like this). I've also seen that when working with a global team, offering different time zone friendly times will allow different people attend each month.
I know Warehouse did stuff like this, so maybe their experience will help getting something that works well here too. But again, it's not time zones that make IRC/chat a problem for me, it's the fact that this is a hobby/spare time activity, and reserving a block of time around work, family and other hobbies isn't easy - whereas catching up on emails can be done when it suits me. (I don't know what other people's situations are - for me, this talk of realtime meetings, minutes, etc, feels much more like "real work" than "hobby". I get way too much of meetings in my day job...) Paul

Thanks for kicking off the discussion. I know we are all quite busy so I wanted to keep this short -- I think Donald brings up some good points about the specific topics we might want to focus on:
I think our first order of business should basically be to figure out some sort of long term decision making process[...]
I agree with this sentiment -- it will be difficult to be productive and to act as a unified organization if we don't have any cohesive mechanism for making decisions. Often I see people looking at similar organizations and evaluating the options when establishing a governance model for a new organization. Are there specific organizations we can look at to draw inspiration from?
Two things -- I think some kind of synchronous communication is incredibly helpful, especially for those of us who don't really know each other yet. The formality of email sometimes makes it difficult to develop a rapport, so I am a little worried that without any agreed-upon synchronous communication we will make it hard for people (like myself, as a newcomer to the PyPA) to collaborate and to feel empowered to assert a contrary position. Most of you probably have spoken to me at least once by now and know that I'm not exactly shy, but the rest of you, I'm not sure how I can get to know you personally without the aid of synchronous communication. On the other hand, formal meetings are somewhat problematic themselves because they tend to favor the most dominant voices and I would hate to silence a good idea by never giving it the space to be articulated. And like Paul, I spend most of my day in meetings, so I'm not eager to put more of those on my calendar. If there were some way to have a communication platform that supported both synchronous and asynchronous communication, that would be ideal... I know slack and others like it claim to do this, but slack is really built for synchronous communication. I'm not sure if there is a good option in this space Roughly speaking I don't think decisions should be made synchronously without allowing asynchronous input somewhere in the process. -Dan Dan Ryan gh: @techalchemy // e: dan@danryan.co

On 7 June 2018 at 09:59, <dan@danryan.co> wrote:
Right, having a common synchronous chat option (e.g. on python.zulipchat.com) is incredibly useful for folks that don't personally like using Twitter as a global asynchronous water cooler (and there are *heaps* of reasons for folks to prefer to avoid Twitter).
Roughly speaking I don't think decisions should be made synchronously without allowing asynchronous input somewhere in the process.
Agreed (and I feel this especially keenly as someone that's essentially 8 hours out of sync with both the US and Europe, so it's personally and professionally routine for me to have async conversations with 24 hour turn-around times between messages). Synchronous comms are useful for tactical decision making on the scale of hours or days where it doesn't really matter whether or not the conversation is inclusive (e.g. folks working on a targeted project like "Migrate PyPI to Warehouse and pypi.org" where the "Responsible & Accountable" lists are short, any required consultation happened up front, and it's OK for everyone else to just be informed of the outcomes after the decisions have already been made) Synchronous comms also work well for collaborative discussion aimed at helping folks that are currently disagreeing with each other better understand each other's perspective (since you can work through many more active listening iterations in a given amount of time synchronously than you can asynchronously). However, synchronous events are appallingly awful for strategic decision making in globally distributed communities. That's one of the reasons we effectively banned the Python Language Summit from actually making any binding language design decisions after the first couple: the folks missing from the in-person event significantly outnumber those that are present. (The other reason was a lack of search-engine-friendly records of the conversation, and that's a concern that applies to most electronic options as well: Google et al already struggle to give good search results from mailing lists, and they're even worse at finding relevant information in real time chat archives) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi all, sorry for a bit of a delay here. I have a first draft of the governance model I've been working on for the PyPA, and I'd like to solicit some feedback. TL;DR: it's consensus-based governance using a lightweight RFC process. I've included the plaintext below for quote-ability, but it might be easier to read it rendered here: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a Cheers, D. # PyPA Lightweight Governance Model In order to better serve the Python Packaging ecosystem, the following governance model is proposed for the Python Packaging Authority. ## Roles The following defines the various roles that may interact within the PyPA. ### PyPA members Anyone with the commit bit on at least one project in the PyPA organization, which should correspond to everyone on the [PyPA-Committers](https://mail.python.org/mm3/mailman3/lists/pypa-committers.python.org/) mailing list. ### PyPA community Essentially anyone who is interested in PyPA activity and wants to follow along or make proposals. ### Packaging-WG members As described in https://wiki.python.org/psf/PackagingWG ### BDFRN (Benevolent Dictator For Right Now) Dustin Ingram, at the moment. ## Goals Below are the primary goals and non-goals for the PyPA and it's governance model. ### Goals of the PyPA These goals are the primary motivation for the existance of the PyPA. These goals are largely already being carried out, even though most have not been explicitly defined. #### Provide support for existing projects under the PyPA In the event that a given project needs additional support, or no longer has active maintainers, the PyPA will ensure that the given project will continue to be supported for users to the extent by which is necessary #### Foster the creation and acceptance of standards for PyPA projects The PyPA should, as much as possible, strive for standardization and coordination across PyPA projects, primarily though the governance process outlined below. #### Guide decisions which affect multiple PyPA projects The PyPA community (especially PyPA members) should be expected to provide opinions, insight and experience when ecosystem-wide changes are being proposed. #### Accept new projects into the PyPA Either from the community, or created organicaly by the PyPA. This is done via the same governance process. #### Enforce adherence to the CoC uniformly across all projects Generally this means leading by example, but occasionally it may mean more explicit moderation. ### Non-goals of the PyPA These are specific items that are explicitly _not_ goals of the PyPA. #### Determine who is and isn't a PyPA member This is for maintainers of individual projects to decide as they add new maintainers (committers) to their projects. Maintainership of a project that is under the PyPA organization automatically transfers membership in the PyPA. #### Micromanage individual projects development As long as the project is adhering to the Code of Conduct, the PyPA should only concerned with large, ecosystem-wide changes. ### Goals of the PyPA's Governance Model These are new goals which the governance model seeks to make possible. #### Provide a consensus-driven decision-making process This will provide a more structured method of proposing and introducing changes than what currently exists, as well as a means by which PyPA members can decide which proposals will be accepted or rejected. #### Provide a transparent record of proposed, accepted and rejected decisions Via an RFC process (described below), the proposed changes to the PyPA ecosystem will be centralized and standardized. This will allow both PyPA members and the wider PyPA community to understand the rationale, process and resolution of any change to the ecosystem. ## Process: The process for PyPA governance is outlined below: ### Lightweight Decision-Making Process: #### 1. Creation of a Request for Comments (RFC) The RFC is created to propose changes to the PyPA ecosystem, with initial proposal, alternatives and tradeoffs, and submitted as a pull request to the `pypa/rfcs` repository. The structure of the RFC is defined below. #### 2. Comments period During this period, the PyPA community and PyPA members publicly discuss the RFC to raise issues or concerns with the proposal, as comments on the pull request. #### 3. Revision of the RFC by the author(s) to address comments This should be repeated above until "major objections" are fully addressed, or it's clear that there is a fundamental choice to be made. #### 4. Put to vote Vote by the PyPA members (via the PyPA-committers mailing list) to confirm consensus (at least two +1's, more +1's than +0's, and no -1's) over a 5 day period. The voting period should not include any significant changes to the RFC -- this should happen in the previous step, where anyone can participate. Voting should also not be conditional, e.g. "+1 if we make it do X instead of Y". #### 5. Review Once voting has concluded, if the RFC has achieved consensus, it has been approved. Otherwise, if the RFC has not achieved consensus, it goes to the BDFRN for review. At this point the BDFRN has the final say in accepting or rejecting the RFC. Regardless of the outcome of the review, the pull request is then amended to reflect the resulting status, and merged. #### 6. Implementation of the proposed change by maintainers If accepted, various maintainers then make the necessary changes. #### 7. Conversion of the RFC into an Architectural Decision Record (ADR) Once implemented, the document should be updated in order to record what was actually implemented (if it differed from the proposed implementation). ### What requires an RFC: The following are examples of situations where an RFC should be proposed. #### Significant user-facing changes Any change which is apparent to the user (highly visible new features, etc). This should not be for trivial changes, and thus should be used at the project maintainer's discretion (or upon suggestion from the PyPA). #### Significant breaking changes or deprecations Any change which removes existing functionality or introduces a breaking change (e.g., deprecating the `python setup.py register` command). This should not be for trivial changes, and thus should be used at the project maintainer's discretion (or upon suggestion from the PyPA). #### Changes affecting multiple PyPA projects Any change which would require modifications to more than one PyPA project (e.g., implementation of `Description-Content-Type` for Markdown READMEs) #### New tools, systems, or standards Either proposing the creation of a new tool, system, or standard, or to accept a new project into the PyPA organization. #### Changes to PyPA Governance or Policy Anything which would change how the PyPA or it's governance process works. Policy proposals should not be mixed with non-policy proposals (e.g., a proposal like PEP 541 "Package Index Name Retention" would not be mixed with a proposal detailing how PyPI plans to implement said policy). ### Creating a RFC: Below are the key components of an RFC: #### Filename The filename should be sequential, containing the RFC number followed by the title. Example: ``` 001-adopting-python3.md ``` #### Title The title should be brief and to the point, short enough to fit in a commit message. Example: ``` # Title RFC-001: Adopting Python 3 ``` #### Summary The summary should be short and concise, maybe even 140 characters or less. ``` # Summary We decided to upgrade from Python 2 to Python 3 to support Unicode better. ``` #### Context The context describes the state of the system when the decision was made. It also records other external forces that come into play. This should eventually become out of date, but still be contextually relevant. Without this context, the decision record might not make sense when reviewed in hindsight. And usually the implementation of this particular architectural decision will make the context out of date—but this is fine. For example: ``` # Context We are currently using Python 2 everywhere. The users want to start sending us Unicode characters in the API. ``` #### Decision The decision is made up of full sentences, in the active voice. Example: ``` # Decision We will migrate to Python 3 for services X, Y and Z. ``` #### Consequences This section should list all consequences (not just the benefits) to the decision, including what could go wrong. Example: ``` # Consequences We will be able to handle Unicode characters, but we might have to provide backwards-compatible support for some dependencies that haven't been ported to Python 3 yet. ``` #### Status The status section should only be one of the six following statuses. The flow is diagrammed as follows: ``` Proposed -----> Accepted ----> Implemented | | | V V V Rejected Deprecated Superseded ``` ##### Proposed A status of "Proposed" is the initial status of a newly created RFC, during the proposal, comment and consensus period. ##### Accepted A status of "Accepted" is assigned after a vote on the RFC has achieved consensus. ##### Rejected A status of "Rejected" is assigned after a vote on the RFC has failed to achieve consensus. ##### Implemented A status of "Implemented" is assigned after the proposed change has been completed by all necessary maintainers. At the point at which the status changes from "Accepted" to "Implemented", the RFC should be revised to become an Architectural Decision Record (ADR), to accurately describe the true state of the ecosystem after implementation. ##### Deprecated A status of "Deprecated" is assigned to an RFC that was accepted, but either was not or could not be implemented. ##### Superseded A status of "Superseded" is assigned when a new RFC is proposed which oviates a previously implemented ADR. A link should be provided to the decision record which supersedes it: ``` # Status Superseded by [ADR-002](/adrs/002-going-back-to-python-2.md) ``` ##### Links This section should contain relevant links depending on the status of the proposal. Example: ``` # Links Discussion: <https://github.com/pypa/rfcs/pull/42 Vote: <https://mail.python.org/mm3/archives/list/pypa-committers@python.org/thread/...> ``` ## Q&A ### Does this mean that we won't be using the PEP process anymore? For a number of things we currently use the PEP process for, such as defining standards, making changes to package metadata fields, etc., the answer is yes, this governance model is designed to replace that process, with these advantages: * Lower barrier to entry for proposal authors * Wider audience of reviewers * More accessible to the community This doesn't mean that we _can't_ use the PEP process anymore, however, it just limits the scope to changes that require it. This is somewhat subjective, but generally if a proposed change requires modifying CPython itself, it should be a PEP. On Fri, Jun 8, 2018 at 8:39 AM Nick Coghlan <ncoghlan@gmail.com> wrote:

Thanks Dustin :) Pradyun, maybe we could comment directly on the gist? *Nicole Harris* Kabu Creative *Mobile:* + 44 (0) 7480 910 191 *E-mail:* n.harris@kabucreative.com http://kabucreative.com On 8 August 2018 at 07:40, Pradyun Gedam <pradyunsg@gmail.com> wrote:

On Wed, 8 Aug 2018 at 02:44, Dustin Ingram <di@python.org> wrote:
How does this (in particular the discussion on RFCs) relate to packaging standards? You say: """ Does this mean that we won't be using the PEP process anymore? For a number of things we currently use the PEP process for, such as defining standards, making changes to package metadata fields, etc., the answer is yes, this governance model is designed to replace that process """ As the BDFL delegate for packaging, I'm not sure I approve of that :-) More specifically, I *don't* approve of the fact that this document doesn't clearly distinguish between the roles of the PyPA and its BDFRN, and the packaging standards process and its BDFL-delegate. Did I just experience an attempted coup? ;-) More seriously, I think it's extremely important that we do clarify those boundaries. As examples, would PEPs 517 and 518 come under my remit, or yours? What about the current discussions on modifying the compatibility tag mechanisms to allow for Linux distro-specific wheels? (In my view, these are clearly "packaging standards" issues, not "PyPA issues"). On the other hand, I don't see "standardization and coordination across PyPA projects" *outside of* formal packaging standards as being in my remit, nor do I consider proposals to accept a project into the PyPA as being something in my area (except, obviously, as a contributing PyPA member). Nick - as the person with prior experience of being packaging BDFL-delegate, what's your view as to how the two areas should be distinguished? Paul

On Wed, 8 Aug 2018 at 09:54, Paul Moore <p.f.moore@gmail.com> wrote:
More seriously, I think it's extremely important that we do clarify those boundaries.
I should add that I'm more than happy to write something up that more clearly defines the role of the "packaging standards" process (and by extension, my role as packaging BDFL-delegate (we really need a mode concise term!)). It's not something that I think we've needed up to now, but if there's going to be confusion between "what's a PyPA issue" and "what's a packaging standards issue", I'm happy to do so. It may take me a week or to to do so, though. (Nick, I'd want to involve you as an initial reviewer of that document, if that's OK). Personally, there's no real confusion in my mind over where the boundary lies, but Dustin's document makes it pretty clear than not everyone shares this clarity ;-) Paul

On Wed, Aug 8, 2018 at 2:04 AM, Paul Moore <p.f.moore@gmail.com> wrote:
I'm sure we can come up with some distinction, but I wonder: is having two separate processes actually serving a useful purpose? Either way it's basically the same people (i.e., the people reading this), doing the same thing (writing a document, debating it, building consensus, etc.), right, with slightly different set dressing? -n -- Nathaniel J. Smith -- https://vorpus.org

On Wed, 8 Aug 2018 at 10:19, Nathaniel Smith <njs@pobox.com> wrote:
Possibly. I view the standards process as being very much community-focused, and about code interoperability, in the sense that it's about standards that *all* tools (whether PyPA or not) must use to interact with the packaging infrastructure. As such, consensus on distutils-sig is key, much more so, to be honest, than PyPA consensus. Conversely, PyPA governance questions and project standards are of little interest to people outside of the PyPA and discussion can reasonably be focused on PyPA members' opinions. But if people want to say that it's the same process, then we have to establish if it's me or the PyPA BDFRN that is the deciding authority for that process. To give a very specific example, I very strongly want (assuming people aren't telling me that they think I'm doing a bad job!) to remain as the deciding authority for standards like PEP 517 and compatibility tags. But I've no interest in managing PyPA governance proposals. So that's a clear distinction, at least in my mind :-) On the other hand, having the same formal process, just with different subject areas and responsible individuals, is fine with me - I'm not obsessed with bureaucracy, just with results. Paul

Yes, sorry Paul, not trying to stage a coup here. :) My plan was to hand over the BDFRN reigns to you once the responsibilities no longer include "define a governance model". However, if you're not interested in being the arbiter of non-standards related proposals, one thing I had considered but didn't include in the model was: * adding "types" or "tracks" to the proposal (such as "standards") * breaking the BDFRN arbiter role into multiple people, one per type That way you could have the final way in any proposal that is "standards" related, someone else would handle "policy", etc.
I think this model aligns well with that existing focus for standards. Since an RFC is just a pull request, it would be open to anyone in the community to comment on, including (but not limited to) the members of distutils-sig. Also, at least to my knowledge, we're not truly gathering consensus on distutils-sig via a vote, we're just getting a general sense of "nobody strongly dislikes this proposal" which would still be possible by simply sharing a link with that mailing list whenever a new proposal would be of interest to that group. D. On Wed, Aug 8, 2018 at 4:35 AM Paul Moore <p.f.moore@gmail.com> wrote:

I do think there's a reasonable distinction between standards and project governance, but it may be worth noting that without a PyPA-wide governance, we already have two reasonable levels of governance: 1. Standards - This is necessary for interop purposes between PyPA and non-PyPA projects (including things like proprietary mirrors, which are not likely to /ever/ fall under the remit of the PyPA). 2. Per-project governance - The PyPA is a collection of otherwise independent projects, each of which has its own internal standards and governance, etc, and we don't want that to change. In my original draft of this e-mail, I wrote up what I thought the role for a PyPA government would be separate from these two levels, but then I actually /read/✝ Dustin's proposal and realized he did all the work for me already in the goals section: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a#goals To the extent that PyPA governance overlaps with #1, so I don't see any real problems here. If anyone is interested in my own rough formulation of these goals (which has a lot of overlap with Dustin's even though it was developed independently), here are the three main areas of interest I see for PyPA-wide organization: 1. Coordination of communication - Intermittently making sure everyone's on the same page, maybe setting up a common location for meeting minutes, etc. 2. Coordination of funding - For the most part I imagine this will be dealt with in the PSF Packaging Working Group, but whatever PyPA-wide governance model we decide should probably at least be concerned with being aware of PyPA priorities which could be brought ot the PWG. 3. Branding-related decisions - A project is given some additional clout when it is released under the brand of a "PyPA project", and so there is a role for governance in making decisions about how this brand should be applied and what the minimum requirements are for being a PyPA project. For example, maintaining the PyPA code of conduct (and if a PyPA project wants a /different/ code of conduct, decision about whether it is "compatible" with the PyPA CoC). ✝Ok, /skimmed/. On 08/08/2018 08:26 AM, Dustin Ingram wrote:

On Wed, 8 Aug 2018 at 14:00, Paul Ganssle <paul@ganssle.io> wrote:
Yes, if we consider "Standards" as independent of "PyPA governance" to the same extent as "Per-project governance" is, then that matches how I see things. In my previous reply to Dustin, that would categorise "PyPA governance" as the "policy" side of the distinction I drew there.
+1 on all of these. And in my view, none of them really overlap much, if at all, with the "interoperability standards" side of things, which is where I see my role lying. Paul

Funding is strictly the responsibility of the Packaging-WG. I think the main interaction here is "if the PyPA has a proposal that needs funding in some way, it will be brought to the WG once approved".
Yes, there is a slightly grey line here between "things internal to a single project" and "things which affect the larger ecosystem" which is largely subjective. The idea here is that maintainers of a given project will know it when they see it, and take care to bring a proposal to the larger PyPA community when it is appropriate. Otherwise the project is free to operate however they please (aside from CoC adherence).
I should say: it's also fine if we just don't want a governance model because we think the existing process is fine. However, for those that weren't at PyCon this year, one of the outcomes from the packaging mini-summit was the idea that we _might_ need a governance model, hence this proposal. D. On Wed, Aug 8, 2018 at 8:10 AM Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 8 Aug 2018 at 16:39, Dustin Ingram <di@python.org> wrote:
I should say: it's also fine if we just don't want a governance model because we think the existing process is fine.
I'm not averse to a governance model (caveat - I consider what we have right now to be a perfectly valid "governance model", it's just less formalised than the one you proposed), I just don't think it necessarily needs to apply to the interoperability PEP side of things (or rather, I don't think there's evidence that we need to *change* the governance model for that).
As I wasn't there, I can't really comment on that. Did anyone present any reasons why what we currently have (for interoperability specs, specifically, as that's the *only* place I have a concern about what you propose) is insufficient, or is not working? I'm a great fan of "if it ain't broke, don't fix it", but that does mean we need to get feedback if people think it *is* broke :-) Paul

So I’m a big fan of adding a more formalized governance model to the PyPA. I think it will help deal with some more community minded issues that we don’t really have a great person deciding for right now (for example, standardizing communication channels as best as we can!). This fracturing ends up hurting users because while it makes sense to us where the lines are drawn and what goes where, your average user might bounce between 2 or 3 different projects before figuring out where their problem actually lies. Trying to standardize those things as much as we reasonably can will only help every body. When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process: - Having two distinct processes, with basically the same people involved is kind of a drag. You can end up with grey areas where you’re not quite sure what process to follow for what. - The PEP process is really designed for changes to Python itself (and even more specifically, generally to CPython itself). While we’ve used it successfully, there are always these sort of weird square peg round hole situations that we’ve hit over the years. Some of them off the top of my head: - It’s actually pretty difficult to locate the packaging PEPs on PEP 0 unless you already know what they are or you just search for the last names of the usual suspects. - The fields and documentations for writing a PEP don’t really fit well, for example the “Type” for the PEP. We tend to bounce between the three types, because none of them are really a great fit. - Because the process itself is tied to what python-dev does, we’re beholden to the decisions there. While some of us (yourself, myself, Nick, etc) are also Python committers and thus have a say in that process, most of the PyPA members are not, and thus have no real say in the PEP process. - Baked into the *current* PEP process, is the idea of a BDFL, which has obviously historically been Guido. However our “power” (such as it is) never really came from Guido’s blessing, it came from the fact that we collectively develop the tooling that the vast bulk of people use to package, distribute, and install their Python projects. To put another way, inherent in the current PEP process is the idea that the BDFL of Python could tomorrow write a new PEP that says all version numbers are to be written in Roman numerals [1], and that puts us in a place where we have to start rejecting some PEPs in our tooling, while still using the process for other changes, and having to communicate to people which PEPs we ignore and which we don’t [2]. - Guido has taken a “permanent vacation”. That means what the PEP process becomes is totally up in the air right now. Technically speaking there is an argument to be made that no PEP can actually be decided on right now, because there is no BDFL (and in PEP land, the BDFL-Delegate’s power flows from the BDFL, with no BDFL there’s no BDFL-Delegate). Of course I doubt anyways is going to stop us from still using our current process, however what is the PEP process going to become? One of the options currently being discussed on python-committers is that all PEPs get decided by a vote of the core developers of CPython… which obviously doesn’t work for us. If that happens where does that leave us? Do we just further diverge from the “standard” PEP process and retain our BDFL-Delegates? Do we tack a “and for packaging stuff, pypa-committers votes not python-committers” into the PEP process? Even if we stick with a BDFL (or a council of BDFLs) whose to say those members aren’t going to be someone who has Guido’s hands off approach to packaging, and they won’t come and decide that they want to handle packaging PEPs themselves? [3]. That being said, one of the things I might suggest as a change to Dustin’s proposal is the idea of “areas of interest” or something like that. There are three possible ones off the top of my head that I can think of: - Governance / Cross Project decision making - Interoperability Standards - PyPI I think it wouldn’t be a bad idea to allow pypa-committers to basically elect someone to act as the BDFL of a particular area— effectively our own BDFL-Delegate system, but where the power flows from pypa-committers instead of python-committers/a hypothetical future BDFL. It’s unlikely (past the first round of this) that we’d ever elect a delegate for the Governance area of interest, but I think electing ones for the packaging standards, and PyPI would make sense. Dropping the idea of “FL” from this system, having it flow from the pypa-committers gives us an easy fallback for if the delegate for a particular area is unavailable for any reason (steps down, gets hit by a bus, whatever)— we would just fall back to voting by pypa-committers until that person is either available again, or a new delegate is elected. If we went with that, I personally don’t see any reason why we wouldn’t elect to keep the same delegates we currently have (Paul Moore for Interoperability standards, Myself for PyPI) but of course, if it’s now an elected by pypa-committers position, those roles aren’t guaranteed either. [1] Forgive me for the hyperbolically silly example! [2] This has of course always been true, but Guido didn’t really care to step into the packaging space, and we all knew/trusted his decision making anyways, so we knew he wasn’t going to do something like that. [3] This goes back to the idea that the PEP process isn’t our process, it’s CPython’s, and it’s currently got a big ? on what it’s going to look like in the future, and besides a handful of us, the PyPA developers don’t really have a say or input into that at all.

On Wed, 8 Aug 2018 at 18:03, Donald Stufft <donald@stufft.io> wrote:
When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process:
Thanks for the information - I'll have to think it all through before commenting. But there's (I guess) one question that I would like to clarify - and this isn't just for you, it's for the group as a whole. Where do people see my role as "packaging interoperability BDFL delegate" fitting into all of this, both in terms of how things stand currently, and how we want things to be going forward? In the interests of full disclosure, my views are: (Current/past situation) 1. Python-dev (and in particular Guido) had little real interest in packaging, and so the BDFL delegate role was in effect the ultimate authority on packaging questions. 2. It was far more of an enabling role than a decision making one - it was about guiding people to consensus and limiting bikeshedding, not about arbitrating disputes. 3. There was no precedent for Guido, or his delegate, making arbitrary decisions, and the community would likely have just ignored any attempts to do so - suggestions that anyone had the power to arbitrarily declare *anything* as a new standard, have no real basis in fact[1]. 4. It was much wider than PyPA, in the sense that the interop standards are precisely about how *non-PyPA* projects can safely interact with PyPA ones. It also *felt* like a wider role, insofar as "what the PyPA (pip team, warehouse team, ...) thought" always felt like only a part of the discussions. 5. The fact that it linked into the PEP process, while formally important, was a relatively minor admin point in practical terms[2] (Going forward) 1. I want to continue in the role - I feel I can do some good, and I don't think I've had the chance yet to really get involved. 2. I'd prefer not to see authority granted to some form of PyPA voting process. Or to rephrase that, I don't think (as noted above) that there actually *is* any such authority to grant - and I'd like to avoid the appearance (to the wider community) that we think there is. Having said that, I have no problem with (as BDFL-delegate) stating that if I need to decide on the colour of a bikeshed, I'll do so by putting the question to the PyPA voting process. My only constraint would be that I'd expect to retain the right to decide whether to refer to the PyPA or continue looking for consensus[3]. 3. I don't actually think that the fallout in python-dev over Guido's resignation has much impact on us. It might mean that we have to accelerate the process of switching to doing our own admin around signing off and storing standards documents, but that's minor. I doubt that anyone over at python-dev (who isn't already active in the packaging community) will care much either way what we do. 4. I'm not interested in taking on the wider BDFRN role described in Dustin's document. So either we have 2 separate roles, or I'm going to have to relinquish the interop standards role. If that feels like I'm making demands, I'm sorry - I don't intend it to, but I also don't feel like I'd do a good job on the "policy" side of things. Beyond that, though, I'm left in the somewhat uncomfortable position of not being sure whether I'm offering my views on the PyPA governance model, or fighting for my role as interop standards lead. So I think I'd like to wait for a little while before contributing further, to find out what the group as a whole feels should happen to the interop standards lead role in future. Paul [1] If you want an example, I'll happily use my current powers to arbitrarily declare that version numbers must be in roman numerals. I'll bet you any amount of money you like that nothing will change, except possibly the level of credibility I have ;-) [2] And we were working on moving even further in that direction, even before Guido's departure. [3] Before this discussion arose, my intention was to do my best as BDFL-delegate to *never* leave myself in a position where I had to make a substantive decision. If that happened, I'd have considered it a failure on my part in guiding the discussion and consensus process.

tl; dr I was operating under an older proposal, I re-read it and suggested changes. In those changes, the BDFL-Delegate role would in essence be renamed in the new process, PyPA would vote on whether to keep you (and me for PyPI) operating in that role.
So I agree with most of these, and I think that speaks to *why* packaging should have it’s own process that it personally owns. Now, perhaps people disagree with that, but let’s assume for the sake of argument that we think that the concept of packaging in Python should have it’s own process. The first question we would need to answer, is who owns that process? In my opinion, it makes the most sense for the PyPA to own that process, much the same way as the PEP process intermingles the concepts of Python and CPython, this would intermingle the concepts of the PyPA and “Python Packaging”. I don’t think that this is a problem though, because I think we’re all reasonable people and that we care about interoperability not just within our own tooling, but with tooling that for one reason or another, doesn’t fall under the PyPA. Now, you could say it would be more ideologically pure to have a distinct thing that sits above the PyPA, and can represent things that simply aren’t part of the PyPA better. I think going down that road is ultimately a hunt for a spherical cow. The overlap between PyPA and “Python Packaging” is pretty large, and much like how CPython’s PEP process (owned/controlled by python-committers) functions for defining standards for other Python interpreters, the PyPA process would simply act as the place for decision making. Ultimately, part of the reason why we ended up going with the PEP process in the first place is because it was there, and we didn’t have a better answer at the time for “what group of people “owns” Python packaging””. Given a more formal decision making process within the PyPA, I think that we *do* have a better answer for that now, and because of the other changes going on the in the Python ecosystem, it is a good time to transition to fully owning our own processes.
I actually just went and re-read the proposal (I had seen an earlier draft of it, and forgot about one of the changes that Dustin said he was going to make). In my opinion, I think we should nix the BDFRN role (once the final decision of how we govern ourselves has been made). In general I don’t think that we should be applying voting to the vast bulk of these hypothetical RFCs. I would instead of the areas of interest thing that I described above, where the PyPA members elect people to act as the “leader” for that area of interest. Basically, here is the diff of what I would suggest (From Dustin’s proposal): https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d... <https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d...> The *practical* differences between the status quo: * We call proposal RFCs instead of PEPs. * They have a different format, and a slightly difference process (discussion on GitHub Pr, etc). * Instead of BDFL-Delegates, we have “Area of Interest Leader” (Or a better name, I suck at naming). * We add an answer to the question of what happens if we find ourselves without a AoEL / BDFL-Delegate (due to whatever reason). * We add a mechanism to get rid of a AoEL / BDFL-Delegate if the members of the PyPA vote to recall them. The *philosphical* difference between the status quo: * Authority over Python packaging belongs to the PyPA, not to python-committers / the Python BDFL. Assuming for a moment that we adopted my above revision, then basically the first order of business IMO would be to vote to adopt two areas of interests leaders: * Interoperability Standards -> Paul Moore * PyPI / Repository -> Donald Stufft Of course, the PyPA could ultimately decide in that vote not to adopt those either or both of those AoELs, but I personally feel like they make sense? (Although I could just be biased :) ).

I like the updated process but I wonder about having an internal/external policy leader of some kind? Then we can have a triumvirate like whats been discussed on the python-committers list and it has good coverage on all the major fronts I.e. someone to drive a unifying vision and consistent platform, cohesive goals inasmuch as we want to do that, work out how to communicate internally, and facilitate success on all of those fronts I don’t mind voting on that either, there are plenty of people who I feel could do a good job representing the PyPA I want to avoid pre-legislating a lot of things because I think one of python’s real strengths is in not trying to solve problems we don’t have – it forces us to come to the table and talk about them instead of blindly following a process that may have been designed before we understood the kinds of challenges it would need to address. IMHO, the less official structure we set up (“x people have to vote for y amount of time with z period”) the better. I think we can probably come to a consensus about 2-3 people who can make the determination about when to close discussions if necessary, some RFC-like process, and go from there? I like the idea of allowing input, and even a quick survey of approvals, but I’m not sure that demanding unanimity is realistic or anything I’m on board with the general idea and the direction as long as we have some final decisionmakers somewhere and I don’t have to memorize a giant rulebook to participate Dan Ryan gh: @techalchemy <https://github.com/techalchemy> // e: dan@danryan.co From: Donald Stufft [mailto:donald@stufft.io] Sent: Wednesday, August 08, 2018 5:03 PM To: Paul Moore Cc: Dustin Ingram; Paul G; pypa-committers@python.org Subject: [pypa-committers]Re: Proposal: "Lightweight" governance model for PyPA On Aug 8, 2018, at 3:42 PM, Paul Moore <p.f.moore@gmail.com> wrote: On Wed, 8 Aug 2018 at 18:03, Donald Stufft <donald@stufft.io> wrote: When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process: Thanks for the information - I'll have to think it all through before commenting. But there's (I guess) one question that I would like to clarify - and this isn't just for you, it's for the group as a whole. Where do people see my role as "packaging interoperability BDFL delegate" fitting into all of this, both in terms of how things stand currently, and how we want things to be going forward? tl; dr I was operating under an older proposal, I re-read it and suggested changes. In those changes, the BDFL-Delegate role would in essence be renamed in the new process, PyPA would vote on whether to keep you (and me for PyPI) operating in that role. In the interests of full disclosure, my views are: (Current/past situation) 1. Python-dev (and in particular Guido) had little real interest in packaging, and so the BDFL delegate role was in effect the ultimate authority on packaging questions. 2. It was far more of an enabling role than a decision making one - it was about guiding people to consensus and limiting bikeshedding, not about arbitrating disputes. 3. There was no precedent for Guido, or his delegate, making arbitrary decisions, and the community would likely have just ignored any attempts to do so - suggestions that anyone had the power to arbitrarily declare *anything* as a new standard, have no real basis in fact[1]. 4. It was much wider than PyPA, in the sense that the interop standards are precisely about how *non-PyPA* projects can safely interact with PyPA ones. It also *felt* like a wider role, insofar as "what the PyPA (pip team, warehouse team, ...) thought" always felt like only a part of the discussions. 5. The fact that it linked into the PEP process, while formally important, was a relatively minor admin point in practical terms[2] So I agree with most of these, and I think that speaks to *why* packaging should have it’s own process that it personally owns. Now, perhaps people disagree with that, but let’s assume for the sake of argument that we think that the concept of packaging in Python should have it’s own process. The first question we would need to answer, is who owns that process? In my opinion, it makes the most sense for the PyPA to own that process, much the same way as the PEP process intermingles the concepts of Python and CPython, this would intermingle the concepts of the PyPA and “Python Packaging”. I don’t think that this is a problem though, because I think we’re all reasonable people and that we care about interoperability not just within our own tooling, but with tooling that for one reason or another, doesn’t fall under the PyPA. Now, you could say it would be more ideologically pure to have a distinct thing that sits above the PyPA, and can represent things that simply aren’t part of the PyPA better. I think going down that road is ultimately a hunt for a spherical cow. The overlap between PyPA and “Python Packaging” is pretty large, and much like how CPython’s PEP process (owned/controlled by python-committers) functions for defining standards for other Python interpreters, the PyPA process would simply act as the place for decision making. Ultimately, part of the reason why we ended up going with the PEP process in the first place is because it was there, and we didn’t have a better answer at the time for “what group of people “owns” Python packaging””. Given a more formal decision making process within the PyPA, I think that we *do* have a better answer for that now, and because of the other changes going on the in the Python ecosystem, it is a good time to transition to fully owning our own processes. (Going forward) 1. I want to continue in the role - I feel I can do some good, and I don't think I've had the chance yet to really get involved. 2. I'd prefer not to see authority granted to some form of PyPA voting process. Or to rephrase that, I don't think (as noted above) that there actually *is* any such authority to grant - and I'd like to avoid the appearance (to the wider community) that we think there is. Having said that, I have no problem with (as BDFL-delegate) stating that if I need to decide on the colour of a bikeshed, I'll do so by putting the question to the PyPA voting process. My only constraint would be that I'd expect to retain the right to decide whether to refer to the PyPA or continue looking for consensus[3]. 3. I don't actually think that the fallout in python-dev over Guido's resignation has much impact on us. It might mean that we have to accelerate the process of switching to doing our own admin around signing off and storing standards documents, but that's minor. I doubt that anyone over at python-dev (who isn't already active in the packaging community) will care much either way what we do. 4. I'm not interested in taking on the wider BDFRN role described in Dustin's document. So either we have 2 separate roles, or I'm going to have to relinquish the interop standards role. If that feels like I'm making demands, I'm sorry - I don't intend it to, but I also don't feel like I'd do a good job on the "policy" side of things. I actually just went and re-read the proposal (I had seen an earlier draft of it, and forgot about one of the changes that Dustin said he was going to make). In my opinion, I think we should nix the BDFRN role (once the final decision of how we govern ourselves has been made). In general I don’t think that we should be applying voting to the vast bulk of these hypothetical RFCs. I would instead of the areas of interest thing that I described above, where the PyPA members elect people to act as the “leader” for that area of interest. Basically, here is the diff of what I would suggest (From Dustin’s proposal): https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d... The *practical* differences between the status quo: * We call proposal RFCs instead of PEPs. * They have a different format, and a slightly difference process (discussion on GitHub Pr, etc). * Instead of BDFL-Delegates, we have “Area of Interest Leader” (Or a better name, I suck at naming). * We add an answer to the question of what happens if we find ourselves without a AoEL / BDFL-Delegate (due to whatever reason). * We add a mechanism to get rid of a AoEL / BDFL-Delegate if the members of the PyPA vote to recall them. The *philosphical* difference between the status quo: * Authority over Python packaging belongs to the PyPA, not to python-committers / the Python BDFL. Assuming for a moment that we adopted my above revision, then basically the first order of business IMO would be to vote to adopt two areas of interests leaders: * Interoperability Standards -> Paul Moore * PyPI / Repository -> Donald Stufft Of course, the PyPA could ultimately decide in that vote not to adopt those either or both of those AoELs, but I personally feel like they make sense? (Although I could just be biased :) ).

I've made some updates to the document, generally accepting Donald's proposed changes here (with some minor tweaks to the wording), namely: * No BDFRN role as ultimate arbiter * Instead, nominated Interest Area Authorities for various areas of interest under the PyPA, who act as the decision-makers instead of a given proposal being put to a vote * Fallback on a vote if there is no Interest Area Authority or they are otherwise unavailable I've also made some other small suggested miscellaneous changes elsewhere. You can see the diff here: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a/revisions D. On Wed, Aug 8, 2018 at 4:02 PM Donald Stufft <donald@stufft.io> wrote:

On Fri, 10 Aug 2018 at 03:33, Dustin Ingram <di@python.org> wrote:
This thread has touched on a few different areas, so I'm just going to chime in here with a few notes of potential interest: 1. While it was never especially well advertised, https://www.pypa.io/en/latest/specifications/ is the page where I kept track of how we/I had adapted the PEP process to work better for PyPA's purposes. I agree with Donald that it was a workaround though, and it may be desirable to split packaging-only PEPs out to their own process (which can also be used for PyPA governance question), and only use the PEP process for actual standard library changes (such as switching to having setuptools provide the reference implementation of the distutils API, rather than CPython: https://github.com/pypa/packaging-problems/issues/127) 2. https://github.com/pypa/pypa.io/pull/34/files tweaked the current process to account for Guido stepping down as BDFL (the reliance on a BDFL had already been substantially reduced when I transferred my previous responsibilities to Paul) 3. https://github.com/pypa/python-packaging-user-guide/pull/547 finally added the missing link from the active specifications page back to the spec process management page Beyond that, my only objection to the doc is its characterisation of the current PEP process as being inaccessible: following the migration to GitHub last year, the existing PEP-based process is *exactly the same* as the process being proposed from a mechanical perspective, just with a different GitHub repo backing it, and with the "Discussions-To: distutils-sig" being implied by the choice of repo to submit the PR against. The key downsides of moving to a PyPA-only repo are that: 1. we're going to have to find a new set of volunteers to serve as PyPA RFC editors and repo maintainers (rather than being able to share the existing pool of PEP editors); 2. the PyPA process becomes less accessible to the Python community at large, since we're no longer sharing the same process and tools as the reference interpreter implementation (Rust don't have a separate process for cargo, for example - all the core tools use the same system at https://github.com/rust-lang/rfcs ) The main potential benefit I see is one Donald noted earlier in the thread: for folks interested specifically in Python packaging, and not language level questions, PEP 0 doesn't currently take the "Discussions-To" header into account, so the packaging PEPs can be hard to find. That said, I'm also one of the PEP 0/1 maintainers, and I'd be completely open to the idea of enhancing PEP 0 to better expose that categorisation information, as well as potentially amending PEP 1 itself to reframe the "Discussions-To" header as a "Topic Area" or "Interest Area" (and then link the venues to the topics, rather than the other way around). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Just want to reiterate before I respond: it's totally fine if our decision is "we don't actually need a PyPA-specific governance process", but I do think there's value in having our own process.
As (I think) the most recent person to write a packaging-related PEP, as well as a first-time PEP author, I disagree with this based on my personal experience. I think the PEP process is too heavy for the small and incremental changes we often need to make in the Packaging ecosystem. For example, when I first wrote PEP 566, I really was only interested in defining a new metadata version which adds the `Description-Content-Type` field. However, since packaging PEPs are relatively infrequent, and require a not-insignificant amount of work to put together, a lot of other "planned but not yet approved" changes which were unrelated got shoe-horned into the same PEP, such as: * The `Provides-Extra` field: https://www.python.org/dev/peps/pep-0566/#provides-extra-optional-multiple-u... * Changes to version specifiers: https://www.python.org/dev/peps/pep-0566/#version-specifiers * JSON-compatible metadata: https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata Adding the `Provides-Extra` field to the specification added to the work I had to do to implement the PEP, since I also needed to implement support for this field anywhere I wanted to fully support the new metadata version. I was pretty motivated to make this change, so I was happy to take the extra work on. However I think that simply the existence of so many small changes that were "pending PEP" is proof that the PEP process and the way the PyPA currently uses it makes it somewhat inaccessible to make these small yet meaningful changes -- otherwise, they wouldn't have been "pending PEP". Had the proposed process been in place, each of these changes could have been their own RFC, authored by the people who actually wanted them, and could have been accepted long I came around to make the `Description-Content-Type` changes. D. On Sat, Aug 18, 2018 at 12:30 AM Nick Coghlan <ncoghlan@gmail.com> wrote:

On Tue, 28 Aug 2018 at 03:49, Dustin Ingram <di@python.org> wrote:
They were "pending PEP" mainly because it took a long time to unwind the misstep that was PEP 426 - that suffered from classic "second system syndrome", where I treated it as an opportunity for a ground-up redesign of the packaging metadata system, rather than the minimal-and-incremental updates that Daniel Holth originally requested (essentially just the Provides-Extra that was added to PEP 566). If we'd had the idea of adding a canonical transformation to-and-from JSON (ala [1]) back when Donald first objected to continuing with the existing email-header-style metadata format, then we likely could have avoided a time consuming detour through that from-scratch redesign that turned out to involve far too much churn to ever be implementable. However, they were also "pending PEP" *because I/we had previously taken the metadata specifications completely out of the PEP process*: those additional fields *were* already documented at https://packaging.python.org/specifications/core-metadata/ prior to PEP 566, but the artifacts creating them were just PRs against the Python Packaging User Guide (together with the related distutils-sig mailing list discussions). It turned out that didn't provide enough traceability in the decision making process - "this field was added by this PR on this date" just didn't provide the same kind of clarity in communication as "this field was added in this metadata version by this PEP". Thus the hybrid process that PEP 566 established: clarifications can just be a PR, but new fields require a short PEP to explicitly define and approve them, and to bump the metadata version. Those new PEPs won't have a 5 year backlog of "deferred waiting for metadata 2.0" proposals to deal with, and they mostly won't have to deal with changing the spec management process either - they can just add the field that the PEP author cares about. (The "mostly" qualifier covers some of the older interoperability PEPs where the PEP itself currently still serves as the reference documentation - their next updates will need to cover migrating them to packaging.python.org the same way that PEP 566 did for the core metadata)
And if I'd proposed in PEP 426 what was eventually proposed in PEP 566, they could have done the same thing :) Cheers, Nick. [1] https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata P.S. Before CPython moved the main PEPs repo to GitHub, there were real usability issues with the PEP process, hence why we had https://github.com/pypa/interoperability-peps for years. Now though, PEPs work exactly the same way as your proposed RFC process does. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

While I do think the PEP process has gotten *better*, I also do agree with Dustin it is more inaccessible than the proposed process. It might be more antagonistic than required though to call out the PEP process as inaccessible though (and opens up the question of if it’s inaccessible, why not fix it?). A larger reason for me is that by doing our own thing, we can better optimize the process to our personal needs, rather than having the PEP process serve two distinct groups. This is particularly note worthy with Guido stepping down, as python-dev is looking at what their process is going to be in the future, and AFAIK has not even considered the impact on us in their decision making— which is perfectly fine! It just underscores in my mind that we’d be better served by something dedicated to us, whose primary goal is making our decisions.
I don’t think that we even need this really. As a set of text files it doesn’t really need “maintained” the same way software does. There’s no special parser or anything that comes with it like the PEP repository does. It’s just some text files in a repository, and Github handles the Markdown rendering for us. Honestly, I don’t even think *Python* really needs this any more. It made a lot more sense when getting a PEP added to the repository and changes integrated meant emailing a patch, but with a PR based workflow whoever is accepting it can just hit the merge button. There’s not really a large need for some arbiter to sit there and garden the PEPs/RFCs. The primary benefit they bring in 2018 I think is in assigning PEP numbers, the proposed process has the same bottle neck, rust works around it by just using the PR number as the RFC number, we might want to consider doing that by default as well.
I don’t think the community at large uses the PEP process or finds them more or less accessible to anything, because as far as they’re concerned they’re just convenient ways to specify some change that python-dev is making, and they don’t get involved in the process at all. I don’t think that the proposed process really changes much for them.

See here for the continuation (and hopefully conclusion) of this discussion: https://discuss.python.org/t/closing-the-loop-on-pypa-governance-bdfrn/1776 D. On Thu, Aug 30, 2018 at 7:03 PM Donald Stufft <donald@stufft.io> wrote:

On Wed, 8 Aug 2018 at 13:26, Dustin Ingram <di@python.org> wrote:
:-) No problem - the main thing I was really picking up on was that there's probably multiple subject areas here, and they likely need to be at least enumerated, just to be sure people know what they are buying in to. Like you say below.
That sounds reasonable to me. It would be very easy for something like this to splinter into too many areas, so we'll need to exercise caution here, but certainly I see a strong difference between "policy" and (technical) "standards" at least (both in terms of the content of the subject area, and in terms of the skills needed to be an effective BDFRN). I don't think I've personally got the patience with politics and bureaucracy to be a good fit for the "policy" side of the role (see below for hints of my views on voting processes, for example :-))
Yep, that seems right.
Conversely, we've not traditionally agreed standards by a vote, and I'm not sure it's something I'd be too keen on making into a formal process. It's too easy for people to take the stance "well, I didn't vote for it". At the moment, it's more a case of "you had your chance to express your opinions, so you need to accept the results". Of course, we've generally been fortunate in that most standards discussions have usually converged on a consensus (or if not consensus then at least exhausted acquiescence ;-)) and there's never been a real need for any sort of arbitration between entrenched opponents. For standards discussions at least, I'd like to see that continue, and I see my role as very much being to encourage participants to compromise and agree, and not to decide between choices[1]. I'll shut up now before this starts to sound like a political platform :-) Paul [1] I feel like that was very much Guido's style, and it worked really effectively. While I've no pretensions that my design instincts match Guido's, I feel like the real enabler of that approach is a good sense of community, which doesn't rely on any one individual but is something that can be encouraged in everyone.

On Tue, 5 Jun 2018, 07:03 Ernest W. Durbin III, <ewdurbin@gmail.com> wrote:
Monthly meetings (which can be skipped if there's nothing to discuss) sound good to me. Structure of meeting TBD, but a something along the lines of:
Announcements, Old Business, New Business.
LGTM as is. :) A moderator and minute-taker would be responsible for summarizing the
meeting, collecting project updates, and publishing minutes and updates for archival purposes and public review. Meeting note publication method TBD.
Question: Process to choose these people? Is only one person incharge of both duties? Adherence to the PyPA Code of Conduct would be required for all attendees.
#### For participation:
I initially had some reservations about restricting participation of Maintainers but giving it a bit more thought, these conditions do look reasonable to me. OR
I like this model. I think the TBDs would be up for discussion, if we do decide to go through with this, so, I refrained from providing suggestions for them now.

Comments on the proposal itself below, but I wanted to sort of diverge a bit here first. I think our first order of business should basically be to figure out some sort of long term decision making process. In my eyes, things like some sort of communication (realtime, async, whatever) or any of our other really open questions sort of should generally a back seat to the question of how do we, as a group, make decisions. We currently have the “BDFRN” distinction, but I view that as a fairly adhoc idea, sort of like “emergency powers” where the ultimate goal should be to to define our real decision making process, and then transition us to that. Once we have that, then I think we should start focusing on communication and other structure.
This sounds reasonably OK to me as long as we tune the recurrence correctly. My primary concern is that the PyPA is getting larger (and is likely to continue to do so), and we have members from all across the globe, so scheduling a single time when everyone who wants to participate is able to could prove difficult. To my knowledge we basically have people on complete opposite ends of the world, and everywhere in between now. We likely also want to add in some mechanism to prevent these from becoming very long or trying to boil the ocean (particularly the first couple are likely to be more prone to this).

On 6 June 2018 at 15:02, Donald Stufft <donald@stufft.io> wrote:
Comments on the proposal itself below, but I wanted to sort of diverge a bit here first.
I think our first order of business should basically be to figure out some sort of long term decision making process. In my eyes, things like some sort of communication (realtime, async, whatever) or any of our other really open questions sort of should generally a back seat to the question of how do we, as a group, make decisions. We currently have the “BDFRN” distinction, but I view that as a fairly adhoc idea, sort of like “emergency powers” where the ultimate goal should be to to define our real decision making process, and then transition us to that. Once we have that, then I think we should start focusing on communication and other structure.
Agreed on this.
I'm personally not keen on the key process being a real-time one. My personal open source time is structured in such a way that I'd basically never be able to attend something like this. I assume/hope that there would be some sort of mechanism for whatever was discussed to be available to everyone (ideally by being mailed out - I prefer push-style communications over pull-style "the logs are here, go & look") with a means for people who didn't attend (but are valid "participants" as per the description) to comment/contribute after the fact. But as currently stated, there doesn't seem to be anything like that, which bothers me. My personal preference would be that we stick to an asynchronous means of discussion/communication, like email. I've seen no evidence that real-time discussions as a *general* mechanism[1] produce better results, and I have definitely seen evidence that they leave people unable to attend feeling at least somewhat disenfranchised. But I'm happy to go with the majority, as long as attendance is not necessary in order to be involved and listened to on an equal footing with people who *do* attend. Paul [1] Having a chat-style option for cases where real-time communication is useful would be fine. But I'm not sure I see evidence that the PyPA is big enough or has sufficiently urgent issues to warrant something like that.

The real time meet ups main advantage is getting decisions made faster and having the ability to be more direct answering peoples concerns, and, thus allowing us to be more efficient. I do share Pauls concerns about people being able to attend, with timezones and this organization being primarily volunteer based. I like Ernest's idea of a sync tho. There could possibly be notes posted from those who can't attend prior and the meeting char (we could rotate) could represent and get the discussion started. We could all take turns at being the scribe and push out notes - like Sumana did for the Warehouse syncs. Then we always have a follow up async communication (e.g. like this). I've also seen that when working with a global team, offering different time zone friendly times will allow different people attend each month. Just some thoughts. I like the discussion. Cooper

On 6 June 2018 at 19:07, Cooper Ry Lees <me@cooperlees.com> wrote:
The real time meet ups main advantage is getting decisions made faster and having the ability to be more direct answering peoples concerns, and, thus allowing us to be more efficient. I do share Pauls concerns about people being able to attend, with timezones and this organization being primarily volunteer based.
Faster decisions pretty much conflicts directly with involving everyone :-( Do we actually *have* decisions that we need to make quickly? I'm not sure on pip we ever have...
I like Ernest's idea of a sync tho. There could possibly be notes posted from those who can't attend prior and the meeting char (we could rotate) could represent and get the discussion started. We could all take turns at being the scribe and push out notes - like Sumana did for the Warehouse syncs. Then we always have a follow up async communication (e.g. like this). I've also seen that when working with a global team, offering different time zone friendly times will allow different people attend each month.
I know Warehouse did stuff like this, so maybe their experience will help getting something that works well here too. But again, it's not time zones that make IRC/chat a problem for me, it's the fact that this is a hobby/spare time activity, and reserving a block of time around work, family and other hobbies isn't easy - whereas catching up on emails can be done when it suits me. (I don't know what other people's situations are - for me, this talk of realtime meetings, minutes, etc, feels much more like "real work" than "hobby". I get way too much of meetings in my day job...) Paul

Thanks for kicking off the discussion. I know we are all quite busy so I wanted to keep this short -- I think Donald brings up some good points about the specific topics we might want to focus on:
I think our first order of business should basically be to figure out some sort of long term decision making process[...]
I agree with this sentiment -- it will be difficult to be productive and to act as a unified organization if we don't have any cohesive mechanism for making decisions. Often I see people looking at similar organizations and evaluating the options when establishing a governance model for a new organization. Are there specific organizations we can look at to draw inspiration from?
Two things -- I think some kind of synchronous communication is incredibly helpful, especially for those of us who don't really know each other yet. The formality of email sometimes makes it difficult to develop a rapport, so I am a little worried that without any agreed-upon synchronous communication we will make it hard for people (like myself, as a newcomer to the PyPA) to collaborate and to feel empowered to assert a contrary position. Most of you probably have spoken to me at least once by now and know that I'm not exactly shy, but the rest of you, I'm not sure how I can get to know you personally without the aid of synchronous communication. On the other hand, formal meetings are somewhat problematic themselves because they tend to favor the most dominant voices and I would hate to silence a good idea by never giving it the space to be articulated. And like Paul, I spend most of my day in meetings, so I'm not eager to put more of those on my calendar. If there were some way to have a communication platform that supported both synchronous and asynchronous communication, that would be ideal... I know slack and others like it claim to do this, but slack is really built for synchronous communication. I'm not sure if there is a good option in this space Roughly speaking I don't think decisions should be made synchronously without allowing asynchronous input somewhere in the process. -Dan Dan Ryan gh: @techalchemy // e: dan@danryan.co

On 7 June 2018 at 09:59, <dan@danryan.co> wrote:
Right, having a common synchronous chat option (e.g. on python.zulipchat.com) is incredibly useful for folks that don't personally like using Twitter as a global asynchronous water cooler (and there are *heaps* of reasons for folks to prefer to avoid Twitter).
Roughly speaking I don't think decisions should be made synchronously without allowing asynchronous input somewhere in the process.
Agreed (and I feel this especially keenly as someone that's essentially 8 hours out of sync with both the US and Europe, so it's personally and professionally routine for me to have async conversations with 24 hour turn-around times between messages). Synchronous comms are useful for tactical decision making on the scale of hours or days where it doesn't really matter whether or not the conversation is inclusive (e.g. folks working on a targeted project like "Migrate PyPI to Warehouse and pypi.org" where the "Responsible & Accountable" lists are short, any required consultation happened up front, and it's OK for everyone else to just be informed of the outcomes after the decisions have already been made) Synchronous comms also work well for collaborative discussion aimed at helping folks that are currently disagreeing with each other better understand each other's perspective (since you can work through many more active listening iterations in a given amount of time synchronously than you can asynchronously). However, synchronous events are appallingly awful for strategic decision making in globally distributed communities. That's one of the reasons we effectively banned the Python Language Summit from actually making any binding language design decisions after the first couple: the folks missing from the in-person event significantly outnumber those that are present. (The other reason was a lack of search-engine-friendly records of the conversation, and that's a concern that applies to most electronic options as well: Google et al already struggle to give good search results from mailing lists, and they're even worse at finding relevant information in real time chat archives) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi all, sorry for a bit of a delay here. I have a first draft of the governance model I've been working on for the PyPA, and I'd like to solicit some feedback. TL;DR: it's consensus-based governance using a lightweight RFC process. I've included the plaintext below for quote-ability, but it might be easier to read it rendered here: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a Cheers, D. # PyPA Lightweight Governance Model In order to better serve the Python Packaging ecosystem, the following governance model is proposed for the Python Packaging Authority. ## Roles The following defines the various roles that may interact within the PyPA. ### PyPA members Anyone with the commit bit on at least one project in the PyPA organization, which should correspond to everyone on the [PyPA-Committers](https://mail.python.org/mm3/mailman3/lists/pypa-committers.python.org/) mailing list. ### PyPA community Essentially anyone who is interested in PyPA activity and wants to follow along or make proposals. ### Packaging-WG members As described in https://wiki.python.org/psf/PackagingWG ### BDFRN (Benevolent Dictator For Right Now) Dustin Ingram, at the moment. ## Goals Below are the primary goals and non-goals for the PyPA and it's governance model. ### Goals of the PyPA These goals are the primary motivation for the existance of the PyPA. These goals are largely already being carried out, even though most have not been explicitly defined. #### Provide support for existing projects under the PyPA In the event that a given project needs additional support, or no longer has active maintainers, the PyPA will ensure that the given project will continue to be supported for users to the extent by which is necessary #### Foster the creation and acceptance of standards for PyPA projects The PyPA should, as much as possible, strive for standardization and coordination across PyPA projects, primarily though the governance process outlined below. #### Guide decisions which affect multiple PyPA projects The PyPA community (especially PyPA members) should be expected to provide opinions, insight and experience when ecosystem-wide changes are being proposed. #### Accept new projects into the PyPA Either from the community, or created organicaly by the PyPA. This is done via the same governance process. #### Enforce adherence to the CoC uniformly across all projects Generally this means leading by example, but occasionally it may mean more explicit moderation. ### Non-goals of the PyPA These are specific items that are explicitly _not_ goals of the PyPA. #### Determine who is and isn't a PyPA member This is for maintainers of individual projects to decide as they add new maintainers (committers) to their projects. Maintainership of a project that is under the PyPA organization automatically transfers membership in the PyPA. #### Micromanage individual projects development As long as the project is adhering to the Code of Conduct, the PyPA should only concerned with large, ecosystem-wide changes. ### Goals of the PyPA's Governance Model These are new goals which the governance model seeks to make possible. #### Provide a consensus-driven decision-making process This will provide a more structured method of proposing and introducing changes than what currently exists, as well as a means by which PyPA members can decide which proposals will be accepted or rejected. #### Provide a transparent record of proposed, accepted and rejected decisions Via an RFC process (described below), the proposed changes to the PyPA ecosystem will be centralized and standardized. This will allow both PyPA members and the wider PyPA community to understand the rationale, process and resolution of any change to the ecosystem. ## Process: The process for PyPA governance is outlined below: ### Lightweight Decision-Making Process: #### 1. Creation of a Request for Comments (RFC) The RFC is created to propose changes to the PyPA ecosystem, with initial proposal, alternatives and tradeoffs, and submitted as a pull request to the `pypa/rfcs` repository. The structure of the RFC is defined below. #### 2. Comments period During this period, the PyPA community and PyPA members publicly discuss the RFC to raise issues or concerns with the proposal, as comments on the pull request. #### 3. Revision of the RFC by the author(s) to address comments This should be repeated above until "major objections" are fully addressed, or it's clear that there is a fundamental choice to be made. #### 4. Put to vote Vote by the PyPA members (via the PyPA-committers mailing list) to confirm consensus (at least two +1's, more +1's than +0's, and no -1's) over a 5 day period. The voting period should not include any significant changes to the RFC -- this should happen in the previous step, where anyone can participate. Voting should also not be conditional, e.g. "+1 if we make it do X instead of Y". #### 5. Review Once voting has concluded, if the RFC has achieved consensus, it has been approved. Otherwise, if the RFC has not achieved consensus, it goes to the BDFRN for review. At this point the BDFRN has the final say in accepting or rejecting the RFC. Regardless of the outcome of the review, the pull request is then amended to reflect the resulting status, and merged. #### 6. Implementation of the proposed change by maintainers If accepted, various maintainers then make the necessary changes. #### 7. Conversion of the RFC into an Architectural Decision Record (ADR) Once implemented, the document should be updated in order to record what was actually implemented (if it differed from the proposed implementation). ### What requires an RFC: The following are examples of situations where an RFC should be proposed. #### Significant user-facing changes Any change which is apparent to the user (highly visible new features, etc). This should not be for trivial changes, and thus should be used at the project maintainer's discretion (or upon suggestion from the PyPA). #### Significant breaking changes or deprecations Any change which removes existing functionality or introduces a breaking change (e.g., deprecating the `python setup.py register` command). This should not be for trivial changes, and thus should be used at the project maintainer's discretion (or upon suggestion from the PyPA). #### Changes affecting multiple PyPA projects Any change which would require modifications to more than one PyPA project (e.g., implementation of `Description-Content-Type` for Markdown READMEs) #### New tools, systems, or standards Either proposing the creation of a new tool, system, or standard, or to accept a new project into the PyPA organization. #### Changes to PyPA Governance or Policy Anything which would change how the PyPA or it's governance process works. Policy proposals should not be mixed with non-policy proposals (e.g., a proposal like PEP 541 "Package Index Name Retention" would not be mixed with a proposal detailing how PyPI plans to implement said policy). ### Creating a RFC: Below are the key components of an RFC: #### Filename The filename should be sequential, containing the RFC number followed by the title. Example: ``` 001-adopting-python3.md ``` #### Title The title should be brief and to the point, short enough to fit in a commit message. Example: ``` # Title RFC-001: Adopting Python 3 ``` #### Summary The summary should be short and concise, maybe even 140 characters or less. ``` # Summary We decided to upgrade from Python 2 to Python 3 to support Unicode better. ``` #### Context The context describes the state of the system when the decision was made. It also records other external forces that come into play. This should eventually become out of date, but still be contextually relevant. Without this context, the decision record might not make sense when reviewed in hindsight. And usually the implementation of this particular architectural decision will make the context out of date—but this is fine. For example: ``` # Context We are currently using Python 2 everywhere. The users want to start sending us Unicode characters in the API. ``` #### Decision The decision is made up of full sentences, in the active voice. Example: ``` # Decision We will migrate to Python 3 for services X, Y and Z. ``` #### Consequences This section should list all consequences (not just the benefits) to the decision, including what could go wrong. Example: ``` # Consequences We will be able to handle Unicode characters, but we might have to provide backwards-compatible support for some dependencies that haven't been ported to Python 3 yet. ``` #### Status The status section should only be one of the six following statuses. The flow is diagrammed as follows: ``` Proposed -----> Accepted ----> Implemented | | | V V V Rejected Deprecated Superseded ``` ##### Proposed A status of "Proposed" is the initial status of a newly created RFC, during the proposal, comment and consensus period. ##### Accepted A status of "Accepted" is assigned after a vote on the RFC has achieved consensus. ##### Rejected A status of "Rejected" is assigned after a vote on the RFC has failed to achieve consensus. ##### Implemented A status of "Implemented" is assigned after the proposed change has been completed by all necessary maintainers. At the point at which the status changes from "Accepted" to "Implemented", the RFC should be revised to become an Architectural Decision Record (ADR), to accurately describe the true state of the ecosystem after implementation. ##### Deprecated A status of "Deprecated" is assigned to an RFC that was accepted, but either was not or could not be implemented. ##### Superseded A status of "Superseded" is assigned when a new RFC is proposed which oviates a previously implemented ADR. A link should be provided to the decision record which supersedes it: ``` # Status Superseded by [ADR-002](/adrs/002-going-back-to-python-2.md) ``` ##### Links This section should contain relevant links depending on the status of the proposal. Example: ``` # Links Discussion: <https://github.com/pypa/rfcs/pull/42 Vote: <https://mail.python.org/mm3/archives/list/pypa-committers@python.org/thread/...> ``` ## Q&A ### Does this mean that we won't be using the PEP process anymore? For a number of things we currently use the PEP process for, such as defining standards, making changes to package metadata fields, etc., the answer is yes, this governance model is designed to replace that process, with these advantages: * Lower barrier to entry for proposal authors * Wider audience of reviewers * More accessible to the community This doesn't mean that we _can't_ use the PEP process anymore, however, it just limits the scope to changes that require it. This is somewhat subjective, but generally if a proposed change requires modifying CPython itself, it should be a PEP. On Fri, Jun 8, 2018 at 8:39 AM Nick Coghlan <ncoghlan@gmail.com> wrote:

Thanks Dustin :) Pradyun, maybe we could comment directly on the gist? *Nicole Harris* Kabu Creative *Mobile:* + 44 (0) 7480 910 191 *E-mail:* n.harris@kabucreative.com http://kabucreative.com On 8 August 2018 at 07:40, Pradyun Gedam <pradyunsg@gmail.com> wrote:

On Wed, 8 Aug 2018 at 02:44, Dustin Ingram <di@python.org> wrote:
How does this (in particular the discussion on RFCs) relate to packaging standards? You say: """ Does this mean that we won't be using the PEP process anymore? For a number of things we currently use the PEP process for, such as defining standards, making changes to package metadata fields, etc., the answer is yes, this governance model is designed to replace that process """ As the BDFL delegate for packaging, I'm not sure I approve of that :-) More specifically, I *don't* approve of the fact that this document doesn't clearly distinguish between the roles of the PyPA and its BDFRN, and the packaging standards process and its BDFL-delegate. Did I just experience an attempted coup? ;-) More seriously, I think it's extremely important that we do clarify those boundaries. As examples, would PEPs 517 and 518 come under my remit, or yours? What about the current discussions on modifying the compatibility tag mechanisms to allow for Linux distro-specific wheels? (In my view, these are clearly "packaging standards" issues, not "PyPA issues"). On the other hand, I don't see "standardization and coordination across PyPA projects" *outside of* formal packaging standards as being in my remit, nor do I consider proposals to accept a project into the PyPA as being something in my area (except, obviously, as a contributing PyPA member). Nick - as the person with prior experience of being packaging BDFL-delegate, what's your view as to how the two areas should be distinguished? Paul

On Wed, 8 Aug 2018 at 09:54, Paul Moore <p.f.moore@gmail.com> wrote:
More seriously, I think it's extremely important that we do clarify those boundaries.
I should add that I'm more than happy to write something up that more clearly defines the role of the "packaging standards" process (and by extension, my role as packaging BDFL-delegate (we really need a mode concise term!)). It's not something that I think we've needed up to now, but if there's going to be confusion between "what's a PyPA issue" and "what's a packaging standards issue", I'm happy to do so. It may take me a week or to to do so, though. (Nick, I'd want to involve you as an initial reviewer of that document, if that's OK). Personally, there's no real confusion in my mind over where the boundary lies, but Dustin's document makes it pretty clear than not everyone shares this clarity ;-) Paul

On Wed, Aug 8, 2018 at 2:04 AM, Paul Moore <p.f.moore@gmail.com> wrote:
I'm sure we can come up with some distinction, but I wonder: is having two separate processes actually serving a useful purpose? Either way it's basically the same people (i.e., the people reading this), doing the same thing (writing a document, debating it, building consensus, etc.), right, with slightly different set dressing? -n -- Nathaniel J. Smith -- https://vorpus.org

On Wed, 8 Aug 2018 at 10:19, Nathaniel Smith <njs@pobox.com> wrote:
Possibly. I view the standards process as being very much community-focused, and about code interoperability, in the sense that it's about standards that *all* tools (whether PyPA or not) must use to interact with the packaging infrastructure. As such, consensus on distutils-sig is key, much more so, to be honest, than PyPA consensus. Conversely, PyPA governance questions and project standards are of little interest to people outside of the PyPA and discussion can reasonably be focused on PyPA members' opinions. But if people want to say that it's the same process, then we have to establish if it's me or the PyPA BDFRN that is the deciding authority for that process. To give a very specific example, I very strongly want (assuming people aren't telling me that they think I'm doing a bad job!) to remain as the deciding authority for standards like PEP 517 and compatibility tags. But I've no interest in managing PyPA governance proposals. So that's a clear distinction, at least in my mind :-) On the other hand, having the same formal process, just with different subject areas and responsible individuals, is fine with me - I'm not obsessed with bureaucracy, just with results. Paul

Yes, sorry Paul, not trying to stage a coup here. :) My plan was to hand over the BDFRN reigns to you once the responsibilities no longer include "define a governance model". However, if you're not interested in being the arbiter of non-standards related proposals, one thing I had considered but didn't include in the model was: * adding "types" or "tracks" to the proposal (such as "standards") * breaking the BDFRN arbiter role into multiple people, one per type That way you could have the final way in any proposal that is "standards" related, someone else would handle "policy", etc.
I think this model aligns well with that existing focus for standards. Since an RFC is just a pull request, it would be open to anyone in the community to comment on, including (but not limited to) the members of distutils-sig. Also, at least to my knowledge, we're not truly gathering consensus on distutils-sig via a vote, we're just getting a general sense of "nobody strongly dislikes this proposal" which would still be possible by simply sharing a link with that mailing list whenever a new proposal would be of interest to that group. D. On Wed, Aug 8, 2018 at 4:35 AM Paul Moore <p.f.moore@gmail.com> wrote:

I do think there's a reasonable distinction between standards and project governance, but it may be worth noting that without a PyPA-wide governance, we already have two reasonable levels of governance: 1. Standards - This is necessary for interop purposes between PyPA and non-PyPA projects (including things like proprietary mirrors, which are not likely to /ever/ fall under the remit of the PyPA). 2. Per-project governance - The PyPA is a collection of otherwise independent projects, each of which has its own internal standards and governance, etc, and we don't want that to change. In my original draft of this e-mail, I wrote up what I thought the role for a PyPA government would be separate from these two levels, but then I actually /read/✝ Dustin's proposal and realized he did all the work for me already in the goals section: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a#goals To the extent that PyPA governance overlaps with #1, so I don't see any real problems here. If anyone is interested in my own rough formulation of these goals (which has a lot of overlap with Dustin's even though it was developed independently), here are the three main areas of interest I see for PyPA-wide organization: 1. Coordination of communication - Intermittently making sure everyone's on the same page, maybe setting up a common location for meeting minutes, etc. 2. Coordination of funding - For the most part I imagine this will be dealt with in the PSF Packaging Working Group, but whatever PyPA-wide governance model we decide should probably at least be concerned with being aware of PyPA priorities which could be brought ot the PWG. 3. Branding-related decisions - A project is given some additional clout when it is released under the brand of a "PyPA project", and so there is a role for governance in making decisions about how this brand should be applied and what the minimum requirements are for being a PyPA project. For example, maintaining the PyPA code of conduct (and if a PyPA project wants a /different/ code of conduct, decision about whether it is "compatible" with the PyPA CoC). ✝Ok, /skimmed/. On 08/08/2018 08:26 AM, Dustin Ingram wrote:

On Wed, 8 Aug 2018 at 14:00, Paul Ganssle <paul@ganssle.io> wrote:
Yes, if we consider "Standards" as independent of "PyPA governance" to the same extent as "Per-project governance" is, then that matches how I see things. In my previous reply to Dustin, that would categorise "PyPA governance" as the "policy" side of the distinction I drew there.
+1 on all of these. And in my view, none of them really overlap much, if at all, with the "interoperability standards" side of things, which is where I see my role lying. Paul

Funding is strictly the responsibility of the Packaging-WG. I think the main interaction here is "if the PyPA has a proposal that needs funding in some way, it will be brought to the WG once approved".
Yes, there is a slightly grey line here between "things internal to a single project" and "things which affect the larger ecosystem" which is largely subjective. The idea here is that maintainers of a given project will know it when they see it, and take care to bring a proposal to the larger PyPA community when it is appropriate. Otherwise the project is free to operate however they please (aside from CoC adherence).
I should say: it's also fine if we just don't want a governance model because we think the existing process is fine. However, for those that weren't at PyCon this year, one of the outcomes from the packaging mini-summit was the idea that we _might_ need a governance model, hence this proposal. D. On Wed, Aug 8, 2018 at 8:10 AM Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 8 Aug 2018 at 16:39, Dustin Ingram <di@python.org> wrote:
I should say: it's also fine if we just don't want a governance model because we think the existing process is fine.
I'm not averse to a governance model (caveat - I consider what we have right now to be a perfectly valid "governance model", it's just less formalised than the one you proposed), I just don't think it necessarily needs to apply to the interoperability PEP side of things (or rather, I don't think there's evidence that we need to *change* the governance model for that).
As I wasn't there, I can't really comment on that. Did anyone present any reasons why what we currently have (for interoperability specs, specifically, as that's the *only* place I have a concern about what you propose) is insufficient, or is not working? I'm a great fan of "if it ain't broke, don't fix it", but that does mean we need to get feedback if people think it *is* broke :-) Paul

So I’m a big fan of adding a more formalized governance model to the PyPA. I think it will help deal with some more community minded issues that we don’t really have a great person deciding for right now (for example, standardizing communication channels as best as we can!). This fracturing ends up hurting users because while it makes sense to us where the lines are drawn and what goes where, your average user might bounce between 2 or 3 different projects before figuring out where their problem actually lies. Trying to standardize those things as much as we reasonably can will only help every body. When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process: - Having two distinct processes, with basically the same people involved is kind of a drag. You can end up with grey areas where you’re not quite sure what process to follow for what. - The PEP process is really designed for changes to Python itself (and even more specifically, generally to CPython itself). While we’ve used it successfully, there are always these sort of weird square peg round hole situations that we’ve hit over the years. Some of them off the top of my head: - It’s actually pretty difficult to locate the packaging PEPs on PEP 0 unless you already know what they are or you just search for the last names of the usual suspects. - The fields and documentations for writing a PEP don’t really fit well, for example the “Type” for the PEP. We tend to bounce between the three types, because none of them are really a great fit. - Because the process itself is tied to what python-dev does, we’re beholden to the decisions there. While some of us (yourself, myself, Nick, etc) are also Python committers and thus have a say in that process, most of the PyPA members are not, and thus have no real say in the PEP process. - Baked into the *current* PEP process, is the idea of a BDFL, which has obviously historically been Guido. However our “power” (such as it is) never really came from Guido’s blessing, it came from the fact that we collectively develop the tooling that the vast bulk of people use to package, distribute, and install their Python projects. To put another way, inherent in the current PEP process is the idea that the BDFL of Python could tomorrow write a new PEP that says all version numbers are to be written in Roman numerals [1], and that puts us in a place where we have to start rejecting some PEPs in our tooling, while still using the process for other changes, and having to communicate to people which PEPs we ignore and which we don’t [2]. - Guido has taken a “permanent vacation”. That means what the PEP process becomes is totally up in the air right now. Technically speaking there is an argument to be made that no PEP can actually be decided on right now, because there is no BDFL (and in PEP land, the BDFL-Delegate’s power flows from the BDFL, with no BDFL there’s no BDFL-Delegate). Of course I doubt anyways is going to stop us from still using our current process, however what is the PEP process going to become? One of the options currently being discussed on python-committers is that all PEPs get decided by a vote of the core developers of CPython… which obviously doesn’t work for us. If that happens where does that leave us? Do we just further diverge from the “standard” PEP process and retain our BDFL-Delegates? Do we tack a “and for packaging stuff, pypa-committers votes not python-committers” into the PEP process? Even if we stick with a BDFL (or a council of BDFLs) whose to say those members aren’t going to be someone who has Guido’s hands off approach to packaging, and they won’t come and decide that they want to handle packaging PEPs themselves? [3]. That being said, one of the things I might suggest as a change to Dustin’s proposal is the idea of “areas of interest” or something like that. There are three possible ones off the top of my head that I can think of: - Governance / Cross Project decision making - Interoperability Standards - PyPI I think it wouldn’t be a bad idea to allow pypa-committers to basically elect someone to act as the BDFL of a particular area— effectively our own BDFL-Delegate system, but where the power flows from pypa-committers instead of python-committers/a hypothetical future BDFL. It’s unlikely (past the first round of this) that we’d ever elect a delegate for the Governance area of interest, but I think electing ones for the packaging standards, and PyPI would make sense. Dropping the idea of “FL” from this system, having it flow from the pypa-committers gives us an easy fallback for if the delegate for a particular area is unavailable for any reason (steps down, gets hit by a bus, whatever)— we would just fall back to voting by pypa-committers until that person is either available again, or a new delegate is elected. If we went with that, I personally don’t see any reason why we wouldn’t elect to keep the same delegates we currently have (Paul Moore for Interoperability standards, Myself for PyPI) but of course, if it’s now an elected by pypa-committers position, those roles aren’t guaranteed either. [1] Forgive me for the hyperbolically silly example! [2] This has of course always been true, but Guido didn’t really care to step into the packaging space, and we all knew/trusted his decision making anyways, so we knew he wasn’t going to do something like that. [3] This goes back to the idea that the PEP process isn’t our process, it’s CPython’s, and it’s currently got a big ? on what it’s going to look like in the future, and besides a handful of us, the PyPA developers don’t really have a say or input into that at all.

On Wed, 8 Aug 2018 at 18:03, Donald Stufft <donald@stufft.io> wrote:
When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process:
Thanks for the information - I'll have to think it all through before commenting. But there's (I guess) one question that I would like to clarify - and this isn't just for you, it's for the group as a whole. Where do people see my role as "packaging interoperability BDFL delegate" fitting into all of this, both in terms of how things stand currently, and how we want things to be going forward? In the interests of full disclosure, my views are: (Current/past situation) 1. Python-dev (and in particular Guido) had little real interest in packaging, and so the BDFL delegate role was in effect the ultimate authority on packaging questions. 2. It was far more of an enabling role than a decision making one - it was about guiding people to consensus and limiting bikeshedding, not about arbitrating disputes. 3. There was no precedent for Guido, or his delegate, making arbitrary decisions, and the community would likely have just ignored any attempts to do so - suggestions that anyone had the power to arbitrarily declare *anything* as a new standard, have no real basis in fact[1]. 4. It was much wider than PyPA, in the sense that the interop standards are precisely about how *non-PyPA* projects can safely interact with PyPA ones. It also *felt* like a wider role, insofar as "what the PyPA (pip team, warehouse team, ...) thought" always felt like only a part of the discussions. 5. The fact that it linked into the PEP process, while formally important, was a relatively minor admin point in practical terms[2] (Going forward) 1. I want to continue in the role - I feel I can do some good, and I don't think I've had the chance yet to really get involved. 2. I'd prefer not to see authority granted to some form of PyPA voting process. Or to rephrase that, I don't think (as noted above) that there actually *is* any such authority to grant - and I'd like to avoid the appearance (to the wider community) that we think there is. Having said that, I have no problem with (as BDFL-delegate) stating that if I need to decide on the colour of a bikeshed, I'll do so by putting the question to the PyPA voting process. My only constraint would be that I'd expect to retain the right to decide whether to refer to the PyPA or continue looking for consensus[3]. 3. I don't actually think that the fallout in python-dev over Guido's resignation has much impact on us. It might mean that we have to accelerate the process of switching to doing our own admin around signing off and storing standards documents, but that's minor. I doubt that anyone over at python-dev (who isn't already active in the packaging community) will care much either way what we do. 4. I'm not interested in taking on the wider BDFRN role described in Dustin's document. So either we have 2 separate roles, or I'm going to have to relinquish the interop standards role. If that feels like I'm making demands, I'm sorry - I don't intend it to, but I also don't feel like I'd do a good job on the "policy" side of things. Beyond that, though, I'm left in the somewhat uncomfortable position of not being sure whether I'm offering my views on the PyPA governance model, or fighting for my role as interop standards lead. So I think I'd like to wait for a little while before contributing further, to find out what the group as a whole feels should happen to the interop standards lead role in future. Paul [1] If you want an example, I'll happily use my current powers to arbitrarily declare that version numbers must be in roman numerals. I'll bet you any amount of money you like that nothing will change, except possibly the level of credibility I have ;-) [2] And we were working on moving even further in that direction, even before Guido's departure. [3] Before this discussion arose, my intention was to do my best as BDFL-delegate to *never* leave myself in a position where I had to make a substantive decision. If that happened, I'd have considered it a failure on my part in guiding the discussion and consensus process.

tl; dr I was operating under an older proposal, I re-read it and suggested changes. In those changes, the BDFL-Delegate role would in essence be renamed in the new process, PyPA would vote on whether to keep you (and me for PyPI) operating in that role.
So I agree with most of these, and I think that speaks to *why* packaging should have it’s own process that it personally owns. Now, perhaps people disagree with that, but let’s assume for the sake of argument that we think that the concept of packaging in Python should have it’s own process. The first question we would need to answer, is who owns that process? In my opinion, it makes the most sense for the PyPA to own that process, much the same way as the PEP process intermingles the concepts of Python and CPython, this would intermingle the concepts of the PyPA and “Python Packaging”. I don’t think that this is a problem though, because I think we’re all reasonable people and that we care about interoperability not just within our own tooling, but with tooling that for one reason or another, doesn’t fall under the PyPA. Now, you could say it would be more ideologically pure to have a distinct thing that sits above the PyPA, and can represent things that simply aren’t part of the PyPA better. I think going down that road is ultimately a hunt for a spherical cow. The overlap between PyPA and “Python Packaging” is pretty large, and much like how CPython’s PEP process (owned/controlled by python-committers) functions for defining standards for other Python interpreters, the PyPA process would simply act as the place for decision making. Ultimately, part of the reason why we ended up going with the PEP process in the first place is because it was there, and we didn’t have a better answer at the time for “what group of people “owns” Python packaging””. Given a more formal decision making process within the PyPA, I think that we *do* have a better answer for that now, and because of the other changes going on the in the Python ecosystem, it is a good time to transition to fully owning our own processes.
I actually just went and re-read the proposal (I had seen an earlier draft of it, and forgot about one of the changes that Dustin said he was going to make). In my opinion, I think we should nix the BDFRN role (once the final decision of how we govern ourselves has been made). In general I don’t think that we should be applying voting to the vast bulk of these hypothetical RFCs. I would instead of the areas of interest thing that I described above, where the PyPA members elect people to act as the “leader” for that area of interest. Basically, here is the diff of what I would suggest (From Dustin’s proposal): https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d... <https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d...> The *practical* differences between the status quo: * We call proposal RFCs instead of PEPs. * They have a different format, and a slightly difference process (discussion on GitHub Pr, etc). * Instead of BDFL-Delegates, we have “Area of Interest Leader” (Or a better name, I suck at naming). * We add an answer to the question of what happens if we find ourselves without a AoEL / BDFL-Delegate (due to whatever reason). * We add a mechanism to get rid of a AoEL / BDFL-Delegate if the members of the PyPA vote to recall them. The *philosphical* difference between the status quo: * Authority over Python packaging belongs to the PyPA, not to python-committers / the Python BDFL. Assuming for a moment that we adopted my above revision, then basically the first order of business IMO would be to vote to adopt two areas of interests leaders: * Interoperability Standards -> Paul Moore * PyPI / Repository -> Donald Stufft Of course, the PyPA could ultimately decide in that vote not to adopt those either or both of those AoELs, but I personally feel like they make sense? (Although I could just be biased :) ).

I like the updated process but I wonder about having an internal/external policy leader of some kind? Then we can have a triumvirate like whats been discussed on the python-committers list and it has good coverage on all the major fronts I.e. someone to drive a unifying vision and consistent platform, cohesive goals inasmuch as we want to do that, work out how to communicate internally, and facilitate success on all of those fronts I don’t mind voting on that either, there are plenty of people who I feel could do a good job representing the PyPA I want to avoid pre-legislating a lot of things because I think one of python’s real strengths is in not trying to solve problems we don’t have – it forces us to come to the table and talk about them instead of blindly following a process that may have been designed before we understood the kinds of challenges it would need to address. IMHO, the less official structure we set up (“x people have to vote for y amount of time with z period”) the better. I think we can probably come to a consensus about 2-3 people who can make the determination about when to close discussions if necessary, some RFC-like process, and go from there? I like the idea of allowing input, and even a quick survey of approvals, but I’m not sure that demanding unanimity is realistic or anything I’m on board with the general idea and the direction as long as we have some final decisionmakers somewhere and I don’t have to memorize a giant rulebook to participate Dan Ryan gh: @techalchemy <https://github.com/techalchemy> // e: dan@danryan.co From: Donald Stufft [mailto:donald@stufft.io] Sent: Wednesday, August 08, 2018 5:03 PM To: Paul Moore Cc: Dustin Ingram; Paul G; pypa-committers@python.org Subject: [pypa-committers]Re: Proposal: "Lightweight" governance model for PyPA On Aug 8, 2018, at 3:42 PM, Paul Moore <p.f.moore@gmail.com> wrote: On Wed, 8 Aug 2018 at 18:03, Donald Stufft <donald@stufft.io> wrote: When it comes to the interoperability specs, there are a few reason why I think we should pull them under the same process: Thanks for the information - I'll have to think it all through before commenting. But there's (I guess) one question that I would like to clarify - and this isn't just for you, it's for the group as a whole. Where do people see my role as "packaging interoperability BDFL delegate" fitting into all of this, both in terms of how things stand currently, and how we want things to be going forward? tl; dr I was operating under an older proposal, I re-read it and suggested changes. In those changes, the BDFL-Delegate role would in essence be renamed in the new process, PyPA would vote on whether to keep you (and me for PyPI) operating in that role. In the interests of full disclosure, my views are: (Current/past situation) 1. Python-dev (and in particular Guido) had little real interest in packaging, and so the BDFL delegate role was in effect the ultimate authority on packaging questions. 2. It was far more of an enabling role than a decision making one - it was about guiding people to consensus and limiting bikeshedding, not about arbitrating disputes. 3. There was no precedent for Guido, or his delegate, making arbitrary decisions, and the community would likely have just ignored any attempts to do so - suggestions that anyone had the power to arbitrarily declare *anything* as a new standard, have no real basis in fact[1]. 4. It was much wider than PyPA, in the sense that the interop standards are precisely about how *non-PyPA* projects can safely interact with PyPA ones. It also *felt* like a wider role, insofar as "what the PyPA (pip team, warehouse team, ...) thought" always felt like only a part of the discussions. 5. The fact that it linked into the PEP process, while formally important, was a relatively minor admin point in practical terms[2] So I agree with most of these, and I think that speaks to *why* packaging should have it’s own process that it personally owns. Now, perhaps people disagree with that, but let’s assume for the sake of argument that we think that the concept of packaging in Python should have it’s own process. The first question we would need to answer, is who owns that process? In my opinion, it makes the most sense for the PyPA to own that process, much the same way as the PEP process intermingles the concepts of Python and CPython, this would intermingle the concepts of the PyPA and “Python Packaging”. I don’t think that this is a problem though, because I think we’re all reasonable people and that we care about interoperability not just within our own tooling, but with tooling that for one reason or another, doesn’t fall under the PyPA. Now, you could say it would be more ideologically pure to have a distinct thing that sits above the PyPA, and can represent things that simply aren’t part of the PyPA better. I think going down that road is ultimately a hunt for a spherical cow. The overlap between PyPA and “Python Packaging” is pretty large, and much like how CPython’s PEP process (owned/controlled by python-committers) functions for defining standards for other Python interpreters, the PyPA process would simply act as the place for decision making. Ultimately, part of the reason why we ended up going with the PEP process in the first place is because it was there, and we didn’t have a better answer at the time for “what group of people “owns” Python packaging””. Given a more formal decision making process within the PyPA, I think that we *do* have a better answer for that now, and because of the other changes going on the in the Python ecosystem, it is a good time to transition to fully owning our own processes. (Going forward) 1. I want to continue in the role - I feel I can do some good, and I don't think I've had the chance yet to really get involved. 2. I'd prefer not to see authority granted to some form of PyPA voting process. Or to rephrase that, I don't think (as noted above) that there actually *is* any such authority to grant - and I'd like to avoid the appearance (to the wider community) that we think there is. Having said that, I have no problem with (as BDFL-delegate) stating that if I need to decide on the colour of a bikeshed, I'll do so by putting the question to the PyPA voting process. My only constraint would be that I'd expect to retain the right to decide whether to refer to the PyPA or continue looking for consensus[3]. 3. I don't actually think that the fallout in python-dev over Guido's resignation has much impact on us. It might mean that we have to accelerate the process of switching to doing our own admin around signing off and storing standards documents, but that's minor. I doubt that anyone over at python-dev (who isn't already active in the packaging community) will care much either way what we do. 4. I'm not interested in taking on the wider BDFRN role described in Dustin's document. So either we have 2 separate roles, or I'm going to have to relinquish the interop standards role. If that feels like I'm making demands, I'm sorry - I don't intend it to, but I also don't feel like I'd do a good job on the "policy" side of things. I actually just went and re-read the proposal (I had seen an earlier draft of it, and forgot about one of the changes that Dustin said he was going to make). In my opinion, I think we should nix the BDFRN role (once the final decision of how we govern ourselves has been made). In general I don’t think that we should be applying voting to the vast bulk of these hypothetical RFCs. I would instead of the areas of interest thing that I described above, where the PyPA members elect people to act as the “leader” for that area of interest. Basically, here is the diff of what I would suggest (From Dustin’s proposal): https://gist.github.com/dstufft/d591cba12abf64818352238ccf05a64e/revisions#d... The *practical* differences between the status quo: * We call proposal RFCs instead of PEPs. * They have a different format, and a slightly difference process (discussion on GitHub Pr, etc). * Instead of BDFL-Delegates, we have “Area of Interest Leader” (Or a better name, I suck at naming). * We add an answer to the question of what happens if we find ourselves without a AoEL / BDFL-Delegate (due to whatever reason). * We add a mechanism to get rid of a AoEL / BDFL-Delegate if the members of the PyPA vote to recall them. The *philosphical* difference between the status quo: * Authority over Python packaging belongs to the PyPA, not to python-committers / the Python BDFL. Assuming for a moment that we adopted my above revision, then basically the first order of business IMO would be to vote to adopt two areas of interests leaders: * Interoperability Standards -> Paul Moore * PyPI / Repository -> Donald Stufft Of course, the PyPA could ultimately decide in that vote not to adopt those either or both of those AoELs, but I personally feel like they make sense? (Although I could just be biased :) ).

I've made some updates to the document, generally accepting Donald's proposed changes here (with some minor tweaks to the wording), namely: * No BDFRN role as ultimate arbiter * Instead, nominated Interest Area Authorities for various areas of interest under the PyPA, who act as the decision-makers instead of a given proposal being put to a vote * Fallback on a vote if there is no Interest Area Authority or they are otherwise unavailable I've also made some other small suggested miscellaneous changes elsewhere. You can see the diff here: https://gist.github.com/di/c785bf9a04f6c5f52d7baf332204f05a/revisions D. On Wed, Aug 8, 2018 at 4:02 PM Donald Stufft <donald@stufft.io> wrote:

On Fri, 10 Aug 2018 at 03:33, Dustin Ingram <di@python.org> wrote:
This thread has touched on a few different areas, so I'm just going to chime in here with a few notes of potential interest: 1. While it was never especially well advertised, https://www.pypa.io/en/latest/specifications/ is the page where I kept track of how we/I had adapted the PEP process to work better for PyPA's purposes. I agree with Donald that it was a workaround though, and it may be desirable to split packaging-only PEPs out to their own process (which can also be used for PyPA governance question), and only use the PEP process for actual standard library changes (such as switching to having setuptools provide the reference implementation of the distutils API, rather than CPython: https://github.com/pypa/packaging-problems/issues/127) 2. https://github.com/pypa/pypa.io/pull/34/files tweaked the current process to account for Guido stepping down as BDFL (the reliance on a BDFL had already been substantially reduced when I transferred my previous responsibilities to Paul) 3. https://github.com/pypa/python-packaging-user-guide/pull/547 finally added the missing link from the active specifications page back to the spec process management page Beyond that, my only objection to the doc is its characterisation of the current PEP process as being inaccessible: following the migration to GitHub last year, the existing PEP-based process is *exactly the same* as the process being proposed from a mechanical perspective, just with a different GitHub repo backing it, and with the "Discussions-To: distutils-sig" being implied by the choice of repo to submit the PR against. The key downsides of moving to a PyPA-only repo are that: 1. we're going to have to find a new set of volunteers to serve as PyPA RFC editors and repo maintainers (rather than being able to share the existing pool of PEP editors); 2. the PyPA process becomes less accessible to the Python community at large, since we're no longer sharing the same process and tools as the reference interpreter implementation (Rust don't have a separate process for cargo, for example - all the core tools use the same system at https://github.com/rust-lang/rfcs ) The main potential benefit I see is one Donald noted earlier in the thread: for folks interested specifically in Python packaging, and not language level questions, PEP 0 doesn't currently take the "Discussions-To" header into account, so the packaging PEPs can be hard to find. That said, I'm also one of the PEP 0/1 maintainers, and I'd be completely open to the idea of enhancing PEP 0 to better expose that categorisation information, as well as potentially amending PEP 1 itself to reframe the "Discussions-To" header as a "Topic Area" or "Interest Area" (and then link the venues to the topics, rather than the other way around). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Just want to reiterate before I respond: it's totally fine if our decision is "we don't actually need a PyPA-specific governance process", but I do think there's value in having our own process.
As (I think) the most recent person to write a packaging-related PEP, as well as a first-time PEP author, I disagree with this based on my personal experience. I think the PEP process is too heavy for the small and incremental changes we often need to make in the Packaging ecosystem. For example, when I first wrote PEP 566, I really was only interested in defining a new metadata version which adds the `Description-Content-Type` field. However, since packaging PEPs are relatively infrequent, and require a not-insignificant amount of work to put together, a lot of other "planned but not yet approved" changes which were unrelated got shoe-horned into the same PEP, such as: * The `Provides-Extra` field: https://www.python.org/dev/peps/pep-0566/#provides-extra-optional-multiple-u... * Changes to version specifiers: https://www.python.org/dev/peps/pep-0566/#version-specifiers * JSON-compatible metadata: https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata Adding the `Provides-Extra` field to the specification added to the work I had to do to implement the PEP, since I also needed to implement support for this field anywhere I wanted to fully support the new metadata version. I was pretty motivated to make this change, so I was happy to take the extra work on. However I think that simply the existence of so many small changes that were "pending PEP" is proof that the PEP process and the way the PyPA currently uses it makes it somewhat inaccessible to make these small yet meaningful changes -- otherwise, they wouldn't have been "pending PEP". Had the proposed process been in place, each of these changes could have been their own RFC, authored by the people who actually wanted them, and could have been accepted long I came around to make the `Description-Content-Type` changes. D. On Sat, Aug 18, 2018 at 12:30 AM Nick Coghlan <ncoghlan@gmail.com> wrote:

On Tue, 28 Aug 2018 at 03:49, Dustin Ingram <di@python.org> wrote:
They were "pending PEP" mainly because it took a long time to unwind the misstep that was PEP 426 - that suffered from classic "second system syndrome", where I treated it as an opportunity for a ground-up redesign of the packaging metadata system, rather than the minimal-and-incremental updates that Daniel Holth originally requested (essentially just the Provides-Extra that was added to PEP 566). If we'd had the idea of adding a canonical transformation to-and-from JSON (ala [1]) back when Donald first objected to continuing with the existing email-header-style metadata format, then we likely could have avoided a time consuming detour through that from-scratch redesign that turned out to involve far too much churn to ever be implementable. However, they were also "pending PEP" *because I/we had previously taken the metadata specifications completely out of the PEP process*: those additional fields *were* already documented at https://packaging.python.org/specifications/core-metadata/ prior to PEP 566, but the artifacts creating them were just PRs against the Python Packaging User Guide (together with the related distutils-sig mailing list discussions). It turned out that didn't provide enough traceability in the decision making process - "this field was added by this PR on this date" just didn't provide the same kind of clarity in communication as "this field was added in this metadata version by this PEP". Thus the hybrid process that PEP 566 established: clarifications can just be a PR, but new fields require a short PEP to explicitly define and approve them, and to bump the metadata version. Those new PEPs won't have a 5 year backlog of "deferred waiting for metadata 2.0" proposals to deal with, and they mostly won't have to deal with changing the spec management process either - they can just add the field that the PEP author cares about. (The "mostly" qualifier covers some of the older interoperability PEPs where the PEP itself currently still serves as the reference documentation - their next updates will need to cover migrating them to packaging.python.org the same way that PEP 566 did for the core metadata)
And if I'd proposed in PEP 426 what was eventually proposed in PEP 566, they could have done the same thing :) Cheers, Nick. [1] https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata P.S. Before CPython moved the main PEPs repo to GitHub, there were real usability issues with the PEP process, hence why we had https://github.com/pypa/interoperability-peps for years. Now though, PEPs work exactly the same way as your proposed RFC process does. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

While I do think the PEP process has gotten *better*, I also do agree with Dustin it is more inaccessible than the proposed process. It might be more antagonistic than required though to call out the PEP process as inaccessible though (and opens up the question of if it’s inaccessible, why not fix it?). A larger reason for me is that by doing our own thing, we can better optimize the process to our personal needs, rather than having the PEP process serve two distinct groups. This is particularly note worthy with Guido stepping down, as python-dev is looking at what their process is going to be in the future, and AFAIK has not even considered the impact on us in their decision making— which is perfectly fine! It just underscores in my mind that we’d be better served by something dedicated to us, whose primary goal is making our decisions.
I don’t think that we even need this really. As a set of text files it doesn’t really need “maintained” the same way software does. There’s no special parser or anything that comes with it like the PEP repository does. It’s just some text files in a repository, and Github handles the Markdown rendering for us. Honestly, I don’t even think *Python* really needs this any more. It made a lot more sense when getting a PEP added to the repository and changes integrated meant emailing a patch, but with a PR based workflow whoever is accepting it can just hit the merge button. There’s not really a large need for some arbiter to sit there and garden the PEPs/RFCs. The primary benefit they bring in 2018 I think is in assigning PEP numbers, the proposed process has the same bottle neck, rust works around it by just using the PR number as the RFC number, we might want to consider doing that by default as well.
I don’t think the community at large uses the PEP process or finds them more or less accessible to anything, because as far as they’re concerned they’re just convenient ways to specify some change that python-dev is making, and they don’t get involved in the process at all. I don’t think that the proposed process really changes much for them.

See here for the continuation (and hopefully conclusion) of this discussion: https://discuss.python.org/t/closing-the-loop-on-pypa-governance-bdfrn/1776 D. On Thu, Aug 30, 2018 at 7:03 PM Donald Stufft <donald@stufft.io> wrote:

On Wed, 8 Aug 2018 at 13:26, Dustin Ingram <di@python.org> wrote:
:-) No problem - the main thing I was really picking up on was that there's probably multiple subject areas here, and they likely need to be at least enumerated, just to be sure people know what they are buying in to. Like you say below.
That sounds reasonable to me. It would be very easy for something like this to splinter into too many areas, so we'll need to exercise caution here, but certainly I see a strong difference between "policy" and (technical) "standards" at least (both in terms of the content of the subject area, and in terms of the skills needed to be an effective BDFRN). I don't think I've personally got the patience with politics and bureaucracy to be a good fit for the "policy" side of the role (see below for hints of my views on voting processes, for example :-))
Yep, that seems right.
Conversely, we've not traditionally agreed standards by a vote, and I'm not sure it's something I'd be too keen on making into a formal process. It's too easy for people to take the stance "well, I didn't vote for it". At the moment, it's more a case of "you had your chance to express your opinions, so you need to accept the results". Of course, we've generally been fortunate in that most standards discussions have usually converged on a consensus (or if not consensus then at least exhausted acquiescence ;-)) and there's never been a real need for any sort of arbitration between entrenched opponents. For standards discussions at least, I'd like to see that continue, and I see my role as very much being to encourage participants to compromise and agree, and not to decide between choices[1]. I'll shut up now before this starts to sound like a political platform :-) Paul [1] I feel like that was very much Guido's style, and it worked really effectively. While I've no pretensions that my design instincts match Guido's, I feel like the real enabler of that approach is a good sense of community, which doesn't rely on any one individual but is something that can be encouraged in everyone.
participants (13)
-
Cooper Ry Lees
-
Dan Ryan
-
dan@danryan.co
-
Donald Stufft
-
Dustin Ingram
-
Ernest W. Durbin III
-
Nathaniel Smith
-
Nick Coghlan
-
Nicole Harris
-
Paul Ganssle
-
Paul Moore
-
Pradyun Gedam
-
Thea Flowers