[Python-Dev] How do we tell if we're helping or hindering the core development process? (was Re: How far to go with user-friendliness)

[Robert Collins]

On 21 July 2015 at 19:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 20 July 2015 at 22:34, Ben Finney <ben+python at benfinney.id.au> wrote:
>> Paul Moore <p.f.moore at gmail.com> writes:
>>
>>> Again, I'm sorry to pick on one sentence out of context, but it cut
>>> straight to my biggest fear when doing a commit (on any project) -
>>> what if, after all the worrying and consideration I put into doing
>>> this commit, people disagree with me (or worse still, I made a
>>> mistake)? Will I be able to justify what I decided?
>>
>> That seems quite healthy to me. On a collaborative project with effects
>> far beyond oneself, yes, any change *should* be able to be justified
>> when challenged.
>
> No, that's not how this works: if folks are thinking that being a
> Python user, or even a CPython core developer, means that we're
> entitled to micromanage core developers by demanding extensive
> explanations for any arbitrary commit we choose, they're thoroughly
> mistaken. Only Guido has that privilege, and one of the reasons he's
> as respected as he is is his willingness to trust the experience and
> expertise of others and only rarely exercise his absolute authority.

I wouldn't even agree that Guido is entitled to micromanage us. He is
certainly entitled to demand explanations etc if he feels appropriate
- but if that turned into micromanaging, I think we'd be in a very bad
place.

> All of this is why the chart that I believe should be worrying people
> is the topmost one on this page:
> http://bugs.python.org/issue?@template=stats
>
> Both the number of open issues and the number of open issues with
> patches are steadily trending upwards. That means the bottleneck in
> the current process *isn't* getting patches written in the first
> place, it's getting them up to the appropriate standards and applied.

Perhaps.

> Yet the answer to the problem isn't a simple "recruit more core
> developers", as the existing core developers are *also* the bottleneck
> in the review and mentoring process for *new* core developers.

sidebar: So, anyone here familiar with theory of constraints - e.g.
'The Goal', 'Critical Chain' etc? Might be interesting to do some
brainstorming in that shape if folk are interest.

Having open idle patches is I think generally acknowledged as a poor
thing. I think of them as inventory in a manufacturing plant: they
take up space, they take effort to create, they offer no value [until
they're actually shipped to users], and they have direct negatives
(tracker is harder to work with due to the signal-to-noise ratio,
perception of the project suffers, contributors make the rational
discussion not to contribute further...).

Lets assume that our actual goal is to ship new Python versions,
offering more and better things to users.

AFAIK we're not lacking any infrastructure resources - we have enough
buildbots, we have freely available compilers for all platforms.

> In my view, those stats are a useful tool we can use to ask ourselves
> "Am I actually helping with this contribution, or at the very least,
> not causing harm?":

I like this approach  :). But - can we go further, can we actively
protect core committer time such that they waste less of it? Adding
core committers won't help if the problem isn't the number of
committers, but rather the amount of the time that they can devote to
Python that actually gets spent on committer-activities.

> * helping core developers that have time to work on "CPython in
> general" rather than specific projects of interest to them to focus
> their attention more effectively may help make those stats better (and
> it would be even better if we could credit such triaging efforts
> appropriately)

Iterating on a patch someone else put up can help. Making sure its
passing tests, trying it with ecosystem projects and giving feedback.

> * exploring ways to extract contribution metrics from Roundup so we
> can have a more reliable detection mechanism for active and consistent
> contributors than the "Mark 1 core developer memory, aka the
> notoriously unreliable human brain" may help make those stats better

OTOH, make sure that what we measure provokes things that help our
goal :). Double edged sword this.

> Make no mistake, sustainable open source development is a *genuinely
> hard problem*. We're in a much better position than most projects
> (being one of the top 5 programming languages in the world has its
> benefits), but we're still effectively running a giant multinational
> collaborative software development project with close to zero formal
> management support. While their are good examples we can (and are)
> learning from, improving the support structures for an already wildly
> successful open source project without alienating existing
> contributors isn't a task that comes with an instruction manual :)

+1

-Rob